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Chapter 


Canada’s Economic Action Plan 


Performance audit reports 


This report presents the results of a performance audit conducted by the Office of 
the Auditor General of Canada under the authority of the Auditor General Act. 


A performance audit is an independent, objective, and systematic assessment 

of how well government is managing its activities, responsibilities, and resources. 
Audit topics are selected based on their significance. While the Office may 
comment on policy implementation in a performance audit, it does not comment 
on the merits of a policy. 


Performance audits are planned, performed, and reported in accordance with 
professional auditing standards and Office policies. They are conducted by 
qualified auditors who 


* establish audit objectives and criteria for the assessment of performance; 
¢ gather the evidence necessary to assess performance against the criteria; 
* report both positive and negative findings; 

¢ conclude against the established audit objectives; and 


* make recommendations for improvement when there are significant 
differences between criteria and assessed performance. 


Performance audits contribute to a public service that is ethical and effective 
and a government that is accountable to Parliament and Canadians. 
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What we examined 
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Main Points 


In January 2009, the Government of Canada launched its Economic 
Action Plan to stimulate the economy in response to the global 
economic downturn. This stimulus plan represented about $47 billion 
in federal spending and an additional $14 billion funded by provinces 
and territories. Its purpose was to create jobs, build infrastructure, 
accelerate housing construction, stimulate spending by Canadians, 
and support businesses and communities. Budget 2009 also contained 
measures to add stability to the financial sector, which sought to 
improve access to financing for consumers and business by providing 
up to $200 billion in credit. 


Our first audit of the Economic Action Plan, reported in October 2010, 
examined program design and delivery mechanisms put in place by 
selected federal departments and agencies to implement the Economic 
Action Plan. 


Our second audit of the Economic Action Plan is the subject of this 
chapter. The audit looked at three programs with a total dollar value 

of $7 billion. The $4 billion Infrastructure Stimulus Fund targeted 
provincial, territorial, and municipal construction-ready projects to build 
or rehabilitate infrastructure. The $2 billion Knowledge Infrastructure 
Program targeted post-secondary institutions across Canada for new 
construction, deferred maintenance, repair, and expansion of projects to 
improve the quality of research and development and deliver advanced 
knowledge and skills training. The $1 billion Community Adjustment 
Fund was established to create or maintain jobs and support businesses 
in communities hit hard by the economic downturn. 


Our audit included two departments and five regional development 
agencies that distributed funds under the programs. We examined 
whether they had monitored and reported on program spending 


and results. 
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Why it’s important 


What we found 


During our audit, the government extended the completion deadline 
for many projects from 31 March 2011 to 31 October 2011. This 
chapter is a report on selected programs up to 31 March 2011, the date 
when our audit work was substantially completed. It is not a report on 
the final results of the Economic Action Plan. 


Given the short time frame planned for the Economic Action Plan, 

as well as the large amounts of public money involved, it was important 
that the government mitigate risks through appropriate monitoring of 
programs at every stage, timely tracking and reporting of costs, 
adherence to its own Policy on Transfer Payments, and reliable 
reporting of results. 


A key risk for the government was that its measures would fail to 
quickly stimulate the Canadian economy. That was the reason for 
giving priority to projects that were “construction ready” and requiring 
that projects be completed by 31 March 2011, to support the 
government’s assertion that stimulus measures would be targeted, 
timely, and temporary. 


- For the three Economic Action Plan programs we audited, the 
federal government monitored the progress and spending of projects, 
permitting it to take corrective action in a number of cases. Progress 
on many projects was slower than initially expected. Departments 
and agencies delivering the three programs that we audited reported 
that 4,070 out of 5,845 projects (70 percent) were completed by 
the 31 March 2011 deadline. Following the federal government’s 
announcement of a deadline extension, almost one third of projects 
in the two largest programs we examined were granted an extension 
to 31 October 2011. The decision to extend the time frames of these 
programs was supported by an analysis conducted by the Privy 
Council Office and the Department of Finance Canada. 


Spending figures provided by the departments and agencies as 
of 31 March 2011 indicate that the three programs had largely 
achieved the Economic Action Plan objective to spend federal 
resources within a two-year time frame. However, total federal 
spending for all three programs will not be known until projects 
submit final claims and close-out reports to the federal entities. 


Although a key objective of the Community Adjustment Fund was 
to create and maintain jobs in communities hit hard by the economic 
downturn, the design of the program did not allow for performance 
measurement and reporting against this key objective. 
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- Infrastructure Canada, Industry Canada, and the regional 
development agencies reported performance information on their 
Economic Action Plan programs in various places throughout their 
departmental performance reports. In our view, this fragmented 
presentation makes it difficult for parliamentarians and Canadians to 
obtain an overall picture of results achieved against planned 
performance expectations and public resources spent. 


The entities have responded. The entities agree with our 
recommendations. Their detailed responses follow each 
recommendation throughout the chapter. 
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CANADA'S ECONOMIC ACTION PLAN 


Introduction 


Background 


11 In January 2009, the federal government introduced a budget to 
mitigate the effects of the severe global economic crisis. The downturn 
was causing significant job losses in key sectors, such as construction 
and forestry. Budget 2009 was named “Canada’s Economic Action 
Plan.” It was intended to stimulate the economy partly through 
increases in government funding for public projects. The plan was 
designed to build infrastructure, stimulate housing construction and 
spending by Canadians, and support businesses and communities. The 
time frame set for the plan was two years. The deadline for completion 
was 31 March 2011 for the vast majority of initiatives. 


1.2 Over 35 federal departments and agencies worked to deliver 
almost 90 programs, including infrastructure programs, in support of 
the Economic Action Plan. The initiatives involved funding of about 
$47 billion, with an additional $14 billion funded by the provinces and 
territories. The plan also provided up to $200 billion in credit to 
improve access to financing for consumers and businesses. The plan 
was based on three guiding principles: that the stimulus should be 
timely, targeted, and temporary. 


1.3. Asa large and complex initiative, the Economic Action Plan 
significantly increased workload in federal departments and agencies. 
Programs were coordinated and delivered through provinces, 
territories, municipalities, third parties, non-governmental 
organizations, and the private sector. Since speed of delivery was 
crucial, officials were under pressure to quickly design, deliver, monitor, 
and report on new or accelerated federal programs, while continuing to 
deliver existing programs. 


Previous audit work 


14 We first examined the 2009 federal stimulus measures in the 
2010 Fall Report of the Auditor General of Canada, Chapter I, 
Canada’s Economic Action Plan. We audited 11 of the Economic 
Action Plan programs through the design and implementation stages. 
We examined program design and delivery, eligibility, the roles of 
internal audit and central agencies, and compliance with financial 


management and environmental requirements. 
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15 To inject $47 billion of stimulus funding into the economy within 
a two-year time frame, the government found it necessary to rely on 
applicants’ attestations that their projects were ready to proceed and 
could be completed by the deadline of 31 March 2011. We found that 
some projects were delayed and that, because the government made 
project approval decisions on the basis of limited information, it had 
accepted the risk that some projects would not meet the deadline. In 
April 2010, we completed our audit work for the chapter included in 
the 2010 Fall Report. At that time, projects were in the early stages of 
implementation, many project files did not contain payment claims or 
progress reports, and it was not known how many projects would be at 
risk of missing the deadline. 


Programs selected for examination 


1.6 For this second audit, we examined selected Economic Action 
Plan programs as they were being delivered and completed. We selected 
three programs that delivered stimulus in the form of thousands of 
projects across the country: the Infrastructure Stimulus Fund, 
administered by Infrastructure Canada; the Knowledge Infrastructure 
Program, administered by Industry Canada; and the Community 
Adjustment Fund, administered by five of the federal government’s 
regional development agencies and Industry Canada (Exhibit 1.1). 


17 The Economic Action Plan allotted $4 billion to the 
Infrastructure Stimulus Fund. This program provided short-term 
funding for construction-ready projects to rehabilitate provincial, 
territorial, and municipal existing assets and new infrastructure. 
Funding was intended to rehabilitate wastewater systems, public 
transit, highways, roads, parks, trails, and municipal buildings. 


18 The Economic Action Plan also allotted $2 billion to the 
Knowledge Infrastructure Program. It supported short-term 
infrastructure enhancement at post-secondary institutions across 
Canada, including universities, colleges, and publicly funded 
polytechnic schools and institutes of technology. Funding was intended 
to support new construction and deferred maintenance, repair, and 
expansion projects aimed at improving research and development 
quality and the ability to deliver advanced knowledge and skills training. 


1.9 The Community Adjustment Fund was allotted $1 billion to 
support activities such as community transition plans that fostered 
economic development, science and technology initiatives, and 
other short-term measures promoting economic diversification. 
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Funding targeted communities that had fewer than 

250,000 residents, had suffered major layoffs, and lacked alternative 
employment opportunities or had experienced a year-over-year 
increase of 20 percent or more in Employment Insurance claimants. 
A primary objective was to maintain existing jobs or create new ones. 


1.10 Recipients of funding under these Economic Action Plan 
programs had to meet several conditions, including requirements to 
complete their projects by 31 March 2011 and to report quarterly on 
project progress. 


Exhibit 1.1 Federal entities and Economic Action Plan programs examined in our second audit 


1A 


Infrastructure Canada Infrastructure Stimulus Fund 4,128 projects 
"$4 billion 
th 
Industry Canada * Knowledge Infrastructure Program 538 projects 
* Community Adjustment Fund, $2 billion 
delivered through Federal Economic 
Development Initiative for Northern 
Ontario* | 
Regional development agencies: * Community Adjustment Fund 1,179 projects 
* Atlantic Canada Opportunities Agency $1 billion 
* Canadian Northern Economic 
Development Agency 


* Economic Development Agency of 
Canada for the Regions of Quebec 


* Federal Economic Development s 
Agency for Southern Ontario 

¢ Western Economic Diversification 
Canada 


Central agencies: ¢ Reporting to Parliament 


* Treasury Board of Canada Secretariat * Extending deadline of selected 
* Department of Finance Canada 
* Privy Council Office 


programs 


* The total allocation and number of approved projects for the Community Adjustment Fund delivered by the Federal Economic Development Initiative for 
Northern Ontario are included in the Community Adjustment Fund totals. 
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Contribution—Conditional transfer payments 
to an individual or organization for a specified 
purpose. These payments are pursuant to a 
contribution agreement, and are subject to being 
accounted for and audited. 


Transfer payment—A monetary payment or a 
transfer of goods, services, or assets to a third 
party. One type of transfer payment is a 
contribution. Transfer payment programs are 
major commitments of federal government 
resources and key instruments for the 
government to further its broad policy objectives 
and priorities. Cabinet determines when a 
transfer payment program is the most 
appropriate policy instrument, and which 
objectives and outcomes are to be achieved by 
means of the transfer payments. 


1.11 For the programs we audited, the federal government transferred 
its funding in the form of contributions. Unlike a grant, the recipient 
of a contribution must give the government an accounting of how it 
spent the money and what results it achieved. 


112 The Infrastructure Stimulus Fund and the Knowledge 
Infrastructure Program transferred federal funding to the provinces 
and territories, as well as to a small number of direct funding recipients, 
including cities, ports, and educational institutions. In most cases, the 
provinces and territories were responsible for monitoring project status, 
verifying claims for payment, and submitting quarterly financial and 
progress reports to the federal department concerned. For the few 
agreements that did not involve a provincial or territorial government, 
Infrastructure Canada and Industry Canada were responsible for 
directly monitoring progress and reviewing claims for eligibility. 


113 Most federal funding under the Community Adjustment 

Fund was transferred directly to individual project recipients through 
contribution agreements administered by five federal regional 
development agencies and, in the case of one region, Industry Canada. 
The project recipients were required to report quarterly on project 
spending and progress, and generally to submit claims to the 
responsible department or regional development agency. 


114 On 2 December 2010, during the planning of this audit, the 
government announced that a small percentage of Economic Action 
Plan projects had been delayed and were at risk of missing 

the 31 March 2011 deadline. The government therefore extended the 
funding and completion deadline for certain Economic Action Plan 
programs by a full construction season to 31 October 2011. This formal 
extension applied to two programs in our audit scope: the Infrastructure 
Stimulus Fund and the Knowledge Infrastructure Program: 


Focus of the audit 


1.15 We audited the Infrastructure Stimulus Fund, the Knowledge 
Infrastructure Program, and the Community Adjustment Fund, which 
had a combined total dollar value of $7 billion. The audit findings 
cannot be extended to other Economic Action Plan programs. Our 
focus was on how the federal government accounted to Parliament and 
Canadians by monitoring project progress, accounting for money 
spent, and reporting on the significant public resources allocated under 
the Economic Action Plan. We did not audit the recipients of 
Economic Action Plan projects or the value received by them as a 
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result of spending under the Plan. Specifically, the audit examined 
whether selected federal departments and agencies had 


* monitored progress, including whether projects were being 
completed as intended, and took corrective action where 
necessary; 


* monitored federal spending for selected Economic Action Plan 
programs; and 


¢ reported to Parliament, through departmental performance 
reports, on progress and actual spending for selected programs. 


116 As part of our examination of monitoring, our audit also 
looked at the role of central agencies in the decision to extend the 
project deadline. 


117 In December 2010, Infrastructure Canada and Industry Canada 
approved new short-term projects, using funds redirected from projects 
that had been approved earlier but that could not be spent by the 
deadline of 31 March 2011. One purpose of approving new projects 
was to minimize the possibility that unspent funding would lapse. 

To be eligible, the new projects had to be completed by 31 March 2011. 
We examined a number of the new projects to confirm their eligibility. 


1.18 For this second audit of the Economic Action Plan, we examined 
projects that were subject to the original deadline of 31 March 2011 as 
well as the extended deadline of 31 October 2011. While we were 
performing the audit, thousands of projects across the country were 
being completed or had been granted an extension. Given the timing 
of the audit and the date set for submitting final close-out reports on 
Economic Action Plan projects, final results were not always available. 


1.19 For projects completed by 31 March 2011, most funding 
recipients were required to submit final close-out documentation 
within 90 days of that date. Since a large number of Infrastructure 
Stimulus Fund projects were expected to be completed close 

to 31 March 2011, Infrastructure Canada gave recipients the option 
to submit final reports by 30 September 2011. For all Infrastructure 
Stimulus Fund and Knowledge Infrastructure Program projects 
extended to 31 October 2011, recipients were required to submit 
final close-out documentation within 90 days of the deadline, that is, 
by 31 January 2012. Until final claims and close-out project reports are 
submitted, departments and agencies will not be able to confirm the 


total federal spending and final project results. 
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Monitoring progress and spending 


Being completed as intended—Refers to 
projects that are progressing in accordance with 
the scope set out in the original application 
approved by the federal government. 


Corrective action—Refers to action that may 
involve withholding federal funds, changing the 
project scope, raising the risk rating, increasing 
monitoring or communications with the recipient, 
extending the project deadline, transferring the 
project to another program, or cancelling 

the project. 


1.20 More details about the audit objectives, scope, approach, and 
criteria are in About the Audit at the end of this chapter. 


Observations and Recommendations 


1.21 In our second audit of the Economic Action Plan, we looked at 
the processes put in place by federal departments and agencies to 
monitor the progress of projects and track government spending. We 
looked at the number of projects that met the original deadline and the 
amount of federal funding spent as of 31 March 2011. Federal 
accountability policies require departments and agencies to monitor 
project progress and federal spending, to confirm whether projects are 
being completed as intended, and to take corrective action as 
necessary. We also looked for evidence that projects approved near the 
end of the Economic Action Plan met eligibility criteria, including the 
requirement to be completed by 31 March 2011. 


1.22 The Treasury Board of Canada Secretariat’s Directive on 
Transfer Payments requires departments and agencies to demonstrate 
accountability, transparency, and effective control in the management 
of transfer payments, such as contributions. Federal program managers 
are required to exercise due diligence in tracking the use of public 
resources; this includes monitoring how funds are spent and to what 
effect. The Directive on Transfer Payments states that departments 
and agencies should respect the jurisdiction of provincial or territorial 
governments. It is the duty of the federal government to have 
accountability mechanisms in place that allow for appropriate and 
timely monitoring and reporting on the use of the funds provided and 
the results achieved. 


1.23 In view of the short time frame of the Economic Action Plan as 
well as the large amount of public money involved, it was important for 
the government to mitigate risks through appropriate monitoring at 
every stage, and to ensure timely tracking of progress and spending. 
Monitoring of federal spending was also important because a key risk 
facing the government was the possible failure of recipients to complete 
projects and spend federal funding by the deadline of 31 March 2011. 
If that happened, the Economic Action Plan might fall short of its 
objective of stimulating the economy in a timely manner. 


124 We reviewed the systems and practices used by departments and 
agencies to monitor overall progress and spending. We also reviewed a 
representative sample of projects funded under the Infrastructure 
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Stimulus Fund, the Knowledge Infrastructure Program, and the 
Community Adjustment Fund. Finally, we conducted site visits to 
selected projects. 


Infrastructure Canada used a risk-based approach to monitor Infrastructure 
Stimulus Fund projects 


1.25 The Infrastructure Stimulus Fund was the largest Economic 
Action Plan infrastructure program. Under this program, Infrastructure 
Canada approved funding for over 4,000 projects in all provinces and 
territories (Exhibit 1.2). Provinces, territories, and some large 
municipalities managed most of the projects. They were responsible for 
monitoring project status, verifying the eligibility of claims, and 
submitting claim and progress reports every three months for projects 
scheduled to end by 31 March 2011. The Department also conducted 
its own monitoring activities. In the case of projects that were extended 
to 31 October 2011, progress reports were required every two months. 


Exhibit 1.2 The Infrastructure Stimulus Fund supported public infrastructure projects 


Highway resurfacing, Burlington, Ontario 


The Infrastructure Stimulus Fund provided 
$4.15 million for a project to replace a stretch 
of a major highway in Burlington, Ontario. The 
aim was to advance the highway’s restoration 
and extend its structural life. 


1.26 The Treasury Board Policy on Transfer Payments requires that 
monitoring and reporting reflect the risks specific to a program and 
the recipient’s risk profile. Consistent with the policy, we found that 
Infrastructure Canada adopted a risk-based approach to monitoring 
Infrastructure Stimulus Fund projects and that it established a process 
including controls at the beginning and end of the program. For 
example, the Department initially assessed risk for every project and 
then used the information provided by its funding partners to reassess 
project risk every quarter. When the Department deemed corrective 
action to be appropriate, it enhanced monitoring for projects and 
redirected funding to new projects from others that were delayed, 
under budget, or cancelled. For medium- and high-risk projects, 
Infrastructure Canada held meetings twice a year. It also conducted 


site visits for high-risk projects. 
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1.27. Under the monitoring approach used by Infrastructure 

Canada, provinces and territories submitted claim and progress reports 
containing information such as key project dates, elapsed time, and 
percentage of project completion. Using this data, as well as 
information obtained through its monitoring activities, Infrastructure 
Canada performed analyses to monitor project progress. For most 
medium- and low-risk projects, the Department relied primarily on the 
monitoring performed by its funding partners to obtain assurance that 
projects were being completed as intended. 


1.28 ‘To assess the risks of relying on funding partners for project 
management, Infrastructure Canada conducted compliance audits. 
These tested whether partners had adequate controls, processes, and 
practices to ensure that recipients complied with the terms and 
conditions of their contribution agreements. The audits were carried 
out in provinces, territories, and two major urban centres midway 
through the two-year delivery schedule for the Infrastructure Stimulus 
Fund. According to Infrastructure Canada officials, the assurances 
provided by these audits confirmed that funding partners had sufficient 
controls in place and were in compliance with the terms of their 
contribution agreements. We did not audit the compliance audits. 


1.29 Provinces and territories reviewed funding recipients’ invoices to 
check eligibility and reimburse costs, and then submitted quarterly 
claim and progress reports to Infrastructure Canada to recover these 
costs. We observed that there could be a considerable lag between the 
time of construction activity and the date when a claim was submitted 
to Infrastructure Canada for reimbursement. To comply with the 
contribution agreement requirement for quarterly reporting, provinces 
and territories often submitted only a progress report, with a zero-dollar 
claim. Sometimes they did not seek reimbursement of costs incurred 
until a project was close to completion. This practice did not always 
provide the Department with current information on project costs 
incurred, but it allowed funding partners (mainly provinces) to cash- 
manage their infrastructure projects. 


1.30 We found that Infrastructure Canada met its requirement to 
monitor the spending of federal funds. The Department used a risk- 
based approach in the review of claims submitted by recipients. The total 
federal contribution is capped; that is, it cannot exceed the maximum 
amount allocated to the program. As of 31 March 2011, the Department 
reported to us that it had paid out approximately $1.7 billion and had 
received information that an additional $1.1 billion in costs had been 
estimated to have been incurred by projects. Some $1.2 billion or 

30 percent of the funding remained to be spent. 


12 


Chapter 1 


Report of the Auditor General of Canada—Fall 2011 


CANADA'S ECONOMIC ACTION PLAN 


1.31 Infrastructure Canada required a final completion report for 
each project, including an attestation by a certified professional that 
the project had been completed as intended. According to preliminary 
information from the Department, approximately two thirds of 

4,128 Infrastructure Stimulus Fund projects were completed as 

of 31 March 2011. The Department also required a final financial 
report for each project to confirm the amount of federal funding spent 
and the eligibility of all costs claimed. Final reports for projects 
completed by 31 March 2011 are due in late September 2011, and in 
January 2012 for projects granted an extension to 31 October 2011. 


1.32. On 20 December 2010, near the end of the two-year period 

of the Economic Action Plan, Infrastructure Canada approved 

42 additional projects on the condition that they be completed 

by 31 March 2011. It funded these Infrastructure Stimulus Fund 
projects using funds from other projects that had been completed 
under budget or cancelled. We looked at a sample of these projects and 
confirmed that they met eligibility criteria. Of the 42 additional 
projects, 34 opted to take advantage of the extension 

to 31 October 2011. 


Additional controls gave Industry Canada increased assurances on Knowledge 
Infrastructure Program projects 


1.33 Under its Knowledge Infrastructure Program, Industry Canada 
approved 538 projects for deferred maintenance, repair, and expansion 
at colleges and universities (Exhibit 1.3). The contribution agreements 
for most of these projects gave the provinces and territories 
responsibility for monitoring them and reviewing the eligibility of 
claims. The Department further required provinces and territories, as 
well as direct recipients, to submit progress reports every three months 
for each project, with information on work performed and outstanding, 
costs incurred, and whether the project was on track to be completed 
by 31 March 2011. For projects that had been granted extensions 

to 31 October 2011, the Department required progress reports every 


two months. 


1.34 We found that Industry Canada monitored the progress of 
projects, tracked federal spending and results, and took corrective 
action when projects were falling behind schedule. To meet its 
accountability requirements for the use of funds and results achieved 
under the Knowledge Infrastructure Program, Industry Canada put a 
number of controls in place. These provided key information to the 
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Department on project progress and federal spending. They also 
helped to manage program risks, including the risk that projects would 


not meet the 31 March 2011 deadline. 


Exhibit 1.3 The Knowledge Infrastructure Program supported projects at academic institutions 


New ventilation system, Mount Allison University, 
New Brunswick 


Industry Canada’s Knowledge Infrastructure Program 
provided $1.2 million to Mount Allison University in 
Sackville, New Brunswick, for a project to install new 
exhaust fans and fume hoods in its chemistry 
building. The equipment replaced an outdated 
ventilation system. The funding recipients told us 
that the project had been a top priority of the 
university for over 10 years. The new system was ; 
designed to provide additional workspaces and cleaner air for professors and students, 
and was expected to lead to more opportunities for performing experiments and 
contributing to research. 


1.35 One of the controls was the design of the Knowledge 
Infrastructure Program’s payment schedule. Industry Canada 
established a regular schedule for payments over the two-year time 
frame, and required projects within the provinces and territories to 
demonstrate satisfactory progress before it would release a payment. 
This requirement gave Industry Canada the information it needed to 
determine whether projects were on track to meet the 31 March 2011 
deadline and whether spending was on target. 


1.36 Another control measure was the hiring of an accounting firm, 
with qualified infrastructure specialists on staff, to review and analyze 
quarterly progress and spending reports from the recipients. The 
reports provided information on total costs incurred, which enabled 
the Department to track the progress of projects against expected 
spending and to identify high-risk projects. Particular attention was 
paid to assessing the progress of projects with a high dollar value. 
The Department then used this information to determine whether it 
should make a regular quarterly payment or take corrective action by 
holding back funding to a province or territory if construction progress 
was behind schedule. 


1.37 We found that the requirement to demonstrate satisfactory 
progress before release of a payment resulted in the frequent 
submission of progress reports on Knowledge Infrastructure Program 
projects. The reports described the work performed and often included 
photographs. The information thus gathered, together with the 
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payment schedule, gave the Department increased assurance that 
projects were being completed as intended and that the federal 
Knowledge Infrastructure Program would stimulate the economy 
within the two-year time frame. 


1.38 The accounting firm conducted site visits and compliance audits 
of selected projects, and reported to Industry Canada on whether 
incurred costs were eligible for reimbursement and whether funds were 
being used as intended. The firm also performed technical reviews of 
some high-risk projects to determine whether they would be able to 
meet the deadline for completion. For projects involving a direct 
agreement between Industry Canada and the recipient, the firm 
performed site visits for the purpose of confirming whether milestones 
were being met and funds were being used for their intended purpose. 
We did not audit the work undertaken by the accounting firm. 


1.39 Like Infrastructure Canada, Industry Canada required a final 
completion report for each project and an attestation by a certified 
professional that the project had been completed as intended. 
Preliminary information indicated that approximately two thirds 

of 538 Knowledge Infrastructure Program projects were completed 
by 31 March 2011. Industry Canada reported to us that as of this date, 
it had spent 88 percent of the $2 billion allocated for the Knowledge 
Infrastructure Program as set out in the payment schedule with the 
provinces and territories. The Department will confirm total federal 
spending for the program as well as eligibility of costs through project 
close-out reports and financial audits. The Department largely 
achieved the Economic Action Plan objective of spending federal 
funding within the two-year time frame. é 


1.40 Like Infrastructure Canada, Industry Canada approved 
additional projects near the end of the two-year period of the 
Economic Action Plan. We found that all of these projects met 
eligibility criteria. The Department reported that all new Knowledge 
Infrastructure Program projects were completed by 31 March 2011. 


Industry Canada and regional agencies monitored Community Adjustment 
Fund projects 


1.41 Five regional development agencies and Industry Canada were 
responsible for delivering the Community Adjustment Fund. They 
approved a total of 1,179 projects that were intended to support 
economic development and create or maintain jobs in communities hit 
hard by the economic downturn (Exhibit 1.4). 
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Exhibit 1.4 The Community Adjustment Fund supported projects in communities hit hard by 
the economic downturn 


New industrial incubator, Drummondville, Quebec 


The Community Adjustment Fund provided 
$500,000 to the Drummondville Economic 
Development Authority to build a multi-sector 
industrial incubator. The facility was designed to 
provide affordable and flexible rental arrangements, 
as well as shared administrative support and 
mentoring for up to seven firms. The project aimed to create new Pier: ibe 
The incubator is expected to help the local community by generating longer-term 
employment and related economic benefits. Without federal assistance, the Authority 
told us that it could not have undertaken the project at this time and that some of the 
new businesses would not be operating. 


Forestry and wildlife project, 
Cranbrook, British Columbia 


The Community Adjustment Fund provided 
approximately $900,000 for a project to restore 
grasslands in Cranbrook. This small B.C. community 
depended heavily on forestry and was severely 
affected when the economic downturn led to the 
shutdown of many local sawmills. The project was 
intended to provide employment, reduce the 
likelihood of forest fires, and restore ecosystems that 
are home to multiple endangered species. 


1.42 We found that regional development agencies and Industry 
Canada tracked the progress of Community Adjustment Fund projects, 
monitored their use of federal funding, confirmed whether projects 
were being completed as intended, and took corrective action when 
necessary. However, we noted variations in the frequency of 
monitoring on the part of regional development agencies and Industry 
Canada. This affected the quality of information available to assess 
project progress, as well as the ability to determine whether projects 
were being completed as intended. 


143 For the projects that we reviewed, Western Economic 
Diversification Canada required recipients to report quarterly on 
progress against project milestones. It then actively monitored progress 
against those milestones and took corrective action when necessaty. 

It also conducted a detailed review of the projects’ claims. This 
approach to monitoring helped identify whether projects were being 
completed on time and as intended. 


144 We found that contribution agreements of the Economic 
Development Agency of Canada for the Regions of Quebec, 
the Atlantic Canada Opportunities Agency, and the Canadian Northern 
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Economic Development Agency did not include a requirement 

that recipients report quarterly on the progress of projects despite the 
terms and conditions of the Community Adjustment Fund program. 
We looked at project files from the Economic Development Agency of 
Canada for the Regions of Quebec and the Atlantic Canada 
Opportunities Agency. While those agencies did not make quarterly 
reporting a requirement, we found that they still collected updates on 
progress and spending for the projects that we reviewed and took 
corrective action when needed. The agencies could request progress 
reporting, but this often involved informal and ongoing communication 
with project recipients, many of whom had an established relationship 
with the agency responsible. The progress information provided agencies 
with increased assurance that projects were being completed on time 
and as intended. 


1.45 In its contribution agreements, the Federal Economic 
Development Agency for Southern Ontario required recipients to 
report monthly against milestones. However, for the projects we 
examined, we found that the Agency did not always enforce this 
reporting requirement. It often did not monitor or verify project 
status until a project was nearly or already completed. As a result, 

the Agency did not know whether some projects were being completed 
as intended until near or after completion of those projects. 


1.46 We also found that approximately one third of Community 
Adjustment Fund projects in Southern Ontario did not meet completion 
deadlines. Two-year funding was not available for projects in this region, 
as the Federal Economic Development Agency for Southern Ontario 
was in the process of being created when the Community Adjustment 
Fund was announced. As a result, projects went through two separate 
project application and approval processes and were funded for one year 
only. Officials told us that the one-year time frame affected the number 
of projects that were completed on time. 


1.47 Most regional development agencies and Industry Canada 
required recipients to submit claims for the costs incurred on their 
projects. We reviewed projects from Western Economic Diversification 
Canada, the Atlantic Canada Opportunities Agency, the Federal 
Economic Development Agency for Southern Ontario, and 

the Economic Development Agency of Canada for the Regions of 
Quebec. We found that these agencies reviewed claims and confirmed 
that costs were eligible before releasing a payment. The review 
provided the agencies with the information they required to ensure 
that recipients were reimbursed only for work that had been performed 
and was eligible under the program. 
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1.48 Preliminary information indicated that approximately 90 percent 
of 1,179 Community Adjustment Fund projects were completed 

by 31 March 2011. The regional development agencies and Industry 
Canada reported to us that, out of the $1 billion allocated to the 
Community Adjustment Fund, they had paid out approximately 

$739 million and had received information that additional costs of 
$142 million had been incurred by projects as of 31 March 2011. This 
means that the regional development agencies and Industry Canada 
spent most (88 percent) of their federal funding within the two-year 
time frame. 


Analysis supported the decision to extend project deadlines 


1.49 Funding under Canada’s Economic Action Plan was designed 
to provide timely, targeted, and temporary stimulus to the Canadian 
economy in order to ensure that the government’s fiscal position was 
structurally sound and to support the recovery. In Budget 2009, the 
government stated that the budget measures were intended to have 
an impact on the economy in the short term and that unspent 
stimulus funds would not be carried forward beyond the end of 

the 2010-11 fiscal year. 


150 Throughout 2009 and 2010, federal monitoring information 
suggested that projects were progressing more slowly than expected 
and the federal government was under increasing pressure to reconsider 
the 31 March 2011 deadline. Parliamentarians and the Federation of 
Canadian Municipalities, among others, called on the government to 
show flexibility and allow more time for projects at risk of missing the 
deadline. In an appearance before the House of Commons Standing 
Committee on Government Operations and Estimates, the Federation 
noted considerable delays in getting projects approved and contribution 
agreements signed; the result, in many cases, was a late start on work 
scheduled to begin during the construction season in the first year of 
the Economic Action Plan. In May 2010, the Standing Committee 
issued an interim report on implementation of the Economic Action 
Plan. The report recommended that the federal government honour its 
funding commitments for all Economic Action Plan projects that had 
been approved for stimulus spending, even if they would not be 


completed by 31 March 2011. 


151 In September 2010, the government responded to the interim 
report of the Standing Committee. It stated that federal officials were 
working with their counterparts across the country to ensure 
completion of projects by 31 March 2011. Furthermore, in Canada’s 
Economic Action Plan, Year 2: A Sixth Report to Canadians 
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(September 2010), the government stated that projects were on track 
to meet the deadline. Despite these assertions and the corrective 
action taken by departments and agencies to mitigate the risk of 
missing the deadline, the federal government continued to receive 
requests for an extension late into 2010. 


1.52. On 2 December 2010, the Prime Minister announced that 

the government would extend the funding deadline by a full 
construction season to 31 October 2011 for four Economic Action 
Plan infrastructure programs, including the Infrastructure Stimulus 
Fund and the Knowledge Infrastructure Program. The Prime Minister 
stated: “Nearly 90 percent of infrastructure projects will be done by 
the ... deadline [31 March 2011] we set out.” 


153 We examined the role of the Privy Council Office and the 
Department of Finance Canada in the decision to offer extensions 

to 31 October 2011 for selected Economic Action Plan infrastructure 
programs. We also looked at the number of infrastructure projects 
given an extension under the Infrastructure Stimulus Fund and the 
Knowledge Infrastructure Program. Finally, we considered how the 
government dealt with Community Adjustment Fund projects that did 


not meet the 31 March 2011 deadline. 


1.54 The implementation of Canada’s Economic Action Plan was a 
priority of the Privy Council Office and the Department of Finance 
Canada. We looked for evidence that these central agencies conducted 
an analysis of the status of projects and the state of the economy in 
support of discussions on whether to extend the project completion 
deadline for selected infrastructure programs. 


~ 


1.55 An informed analysis was important because the Economic 
Action Plan was intended to inject stimulus into the economy, when 
necessary, and withdraw it as the recovery gained a solid footing. 

We looked for evidence that central agencies used data from federal 
departments and agencies on the status of Economic Action Plan 
projects. We also examined whether the government considered the 
risks of stopping federal funding, as well as the risks of extending the 
deadline to the recovery of the economy and to the completion of 


projects that were largely finished. 


1.56 We found that, in October and November 2010, central agencies 
requested regular project status updates and analyses from Infrastructure 
Canada and Industry Canada, assessing the potential amount of federal 
funds that would not be spent by 31 March 2011. Federal officials 
maintained that the vast majority of projects would be completed 
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on time. They were nevertheless aware of a lag between construction 
activities on the ground and reporting of the completed activities; this 
made it difficult to assess precisely the progress of projects. 


1.57 We found that the decision to extend the 31 March 2011 
deadline for projects under the Infrastructure Stimulus Fund and the 
Knowledge Infrastructure Program was based on a central agency 
analysis of information supplied by departments and agencies on the 
progress of projects. We also found that the analysis assessed the risk of 
discontinuing federal funding before Economic Action Plan 
investments could be realized. 


1.58 Furthermore, the central agency analysis considered the strength 
of the economic recovery in Canada and globally. The analysis 
concluded that the weak global economic situation at the time could 
jeopardize the recovery in Canada and that continued federal support 
for Economic Action Plan projects beyond the original deadline was 
appropriate. In 2010, the economy was in a much more fragile state 
than had been expected when the Economic Action Plan was 
launched. The economic environment was not the principal factor 
considered in the decision to change the deadline, but it afforded the 
government the flexibility to extend delayed projects. 


1.59 Infrastructure Canada and Industry Canada reported that 

1,433 Infrastructure Stimulus Fund projects and 189 Knowledge 
Infrastructure Program projects were given an extension 

to 31 October 2011. This represented approximately one third of 
projects in the two programs. The number of extended projects was 
higher than the government had estimated. However, since the 
extension was offered to all recipients able to satisfy new conditions, it 
was not possible to determine conclusively how many projects that opted 
for an extension would have been able to meet the original deadline. 


160 The extension was not offered to Community Adjustment Fund 
projects. According to central agency officials, the original deadline was 
maintained because regional development agency officials had reported 
that the vast majority of projects were on track to be completed, as 
scheduled, by 31 March 2011. When some projects were in fact delayed, 
the agencies, communities, and recipients requested a similar extension. 


161 Regional development agency officials and Industry Canada 
reported that 10 percent of Community Adjustment Fund projects 
were not completed by the original deadline. The government dealt 
with them in a variety of ways. These included granting an additional 
construction season for project completion, transferring the projects to 
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existing programs, and cutting off federal funding and holding the 
recipients responsible for completing projects at their own cost. 


1.62 We noted that a number of factors contributed to delays for 
Community Adjustment Fund projects, including weather problems, 
the need to obtain approvals, and, in the case of the Federal Economic 
Development Agency for Southern Ontario, the one-year time frame 
for project completion. 


Stricter conditions were applied to extended projects 


1.63 To increase assurances that federal stimulus funding would be 
spent by the new deadline of 31 October 2011, recipients requesting 
an extension had to obtain government approval for new project 
conditions. We looked for evidence that extended projects met the 
new conditions, including detailed spending forecasts and construction 
schedules that were signed by a professional engineer or architect. 


1.64 Failure to meet the original 31 March 2011 deadline partly 
reflected the risk of relying solely on applicants’ attestations of project 
readiness and their ability to complete an infrastructure project within 
two years. We had identified this risk in our first audit of the Economic 
Action Plan. The conditions for extension were important to assure 
the government that funding for projects would be spent by the new 
deadline and would thus contribute toward the Economic Action Plan 
objective of stimulating the economy in a timely manner. The deadline 
extension also allowed projects to be completed without losing 

federal funding. 


1.65 We examined a representative sample of extended Infrastructure 
Stimulus Fund and Knowledge Infrastructure Program projects to 
determine whether they complied with the new conditions for 
extension. We found that all projects met the conditions. At the time 
of our audit, most were expected to be completed by 31 October 2011. 


1.66 According to federal officials and recipients, factors such as 
weather, fires, floods, and availability of labour and materials had 
affected the ability to complete projects within the original time frame. 
We also noted that despite attestations of construction readiness, 
municipalities sometimes waited to complete planning and engineering 
work for infrastructure projects until they had secured project funding. 
We further observed that several projects granted an extension were 
large and complex. We visited four of these projects from our 
representative sample, where recipients opted for the extended time 


frame (Exhibit 1.5). 
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Exhibit 1.5 Some large and complex projects were extended 


Expansion of Port of Belledune, New Brunswick 


The Infrastructure Stimulus Fund provided $26.4 million for a project to expand the Port of 
Belledune by adding multi-use cargo storage and constructing a barge terminal. This was a complex 
project involving many partners; according to the funding recipients, without the federal funding, it ® 
would not have been realized for many more years. Delays caused by the weather made it difficult so 
to meet the construction schedule. The deadline was extended to 31 October 2011 to allow the 
recipients additional time to complete the project without compromising construction quality. 


New heating and cooling system, University of Calgary, Alberta 


The Knowledge Infrastructure Program provided $29.8 million for a project to improve the 
university's outdated heating and cooling system. This new system was designed to increase 
energy efficiency and capacity, while decreasing emissions. Unexpected delays in the delivery of 
key equipment parts caused the project to miss the 31 March 2011 deadline for completion. An 
extension was granted and the university expected to complete the project by July 2011. 


New medical building, Queen’s University, Ontario 


The Knowledge Infrastructure Program provided $28.8 million to partly fund the new Queen’s 
School of Medicine. The building houses simulation labs, innovative teaching spaces, teaching 
clinics, student study space, and advanced scientific laboratories. With the new facility, the 
University expects to increase enrolment by 25 percent, and will continue to train highly qualified 
medical professionals and researchers. Unexpected delays in obtaining custom materials put the 
project four to six weeks behind schedule. The only way to meet the original deadline of 31 March 
2011 would have been by paying overtime costs. Instead, the university opted for a two-month 
extension. 


Recreational facilities, Stoney Creek, Ontario 


The Infrastructure Stimulus Fund provided $4.3 million in funding for a project to renovate an 
existing arena and construct a new recreational facility. The project was not construction-ready. 
After funding was granted, it had to obtain municipal permits and approvals as well as undergo 
a complex design phase, a public consultation process, and environmental and archaeological 
studies. The work is expected to continue until summer of 2012. Infrastructure Stimulus Fund 
funding will be provided for the eligible federal portion of costs incurred up to 31 October 2011. 


Program design did not allow for assessment of the key objective of job creation 


1.67 A key objective of the Economic Action Plan was to stimulate 
the economy by creating jobs. In our first audit, we reported that the 
use of project-level information from Economic Action Plan programs 
could not provide a consistent measure of jobs created under the Plan 
overall. Instead, the government intended to rely on an analysis based 
on a macroeconomic model to estimate the number of jobs created or 
maintained. Consequently, central agencies notified departments and 
agencies delivering Economic Action Plan programs that the 
government would not be relying on project-level estimates of jobs 
created as a measure of the plan’s overall success. 
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1.68 As part of our monitoring objective for this audit, we examined 
whether departments and agencies collected information on the progress 
of projects. Under federal policy, departments and agencies should have 
monitored overall progress, including whether recipients were collecting 
and reporting the information required for government officials to 
determine whether a project was being completed as intended. 


1.69 Of the three programs we examined, the Community 
Adjustment Fund was designed to support projects intended to create 
and maintain employment. Collecting information on job creation was 
important because one of the key objectives of the fund was to support 
or create jobs in communities that had experienced significant job 
losses and lacked other employment opportunities for people who 
were out of work. 


1.70 We found that contribution agreements for Community 
Adjustment Fund projects included specific performance indicators 
related to job creation. The performance indicators were designed to be 
used as a measurement of project progress. However, in the absence of 
guidance on how to consistently collect job information, performance 
information was collected in a variety of ways and, in one case, was not 
systematically collected after central agencies announced that job 
information would be used selectively only for illustrative purposes to 
accompany the macroeconomic analysis. The lack of reliable 
performance information on job creation will make it difficult for the 
government to assess the Community Adjustment Fund program’s 
effectiveness in meeting one of its key objectives. 


1.71 Recommendation. When federal initiatives to be delivered by 
multiple federal entities are developed, the sponsoring departments, 
in consultation with the Treasury Board of Canada Secretariat, should 
ensure that programs are designed to allow for reliable performance 
measurement and reporting on overall impact and effectiveness. 


The Department’s and regional development agencies’ response. 
Agreed. When delivering new national programming, Industry Canada 
and regional development agencies will work with federal delivery 
partners and in consultation with the Treasury Board of Canada 
Secretariat to ensure reliable performance measurement and reporting 


on overall program impact and effectiveness. 


The Secretariat’s response. Agreed. As part of its regular business and 
as requested by sponsoring departments, the Secretariat will continue 
to provide advice and guidance on performance measurement and 

reporting in the design of programs delivered by multiple departments. 
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Departmental performance reports for the 2009-10 fiscal year did not provide 
an overall picture of Economic Action Plan program performance 


1.72 Under the Economic Action Plan, departments and agencies 
were required to report on the implementation and effectiveness of 
their Economic Action Plan programs through their departmental 
performance reports. In May 2010, the Secretariat issued guidance 
about reporting on the performance of Economic Action Plan 
programs in departmental performance reports for the 2009-10 fiscal 
year. The reports were to provide information on actual progress and 
spending of Economic Action Plan programs. The first audit of the 
Economic Action Plan recommended that departments delivering 
programs under the Plan report to Parliament by devoting a separate 
section in their annual departmental performance reports to the 
spending and results of those programs. 


1.73 This is important because departmental performance reports are 
intended to provide information to parliamentarians and Canadians 
about results achieved against planned performance expectations and 
public resources spent. The 2009-10 departmental performance 
reports were supposed to provide progress information on how the 
federal government spent $47 billion of public funds for the Economic 
Action Plan. 


1.74 We found that other sources of information on the three audited 
programs were available to parliamentarians. These included 
government reports on the Economic Action Plan, Parliamentary 
Budget Officer reports, department and agency websites, and testimony 
by department officials during parliamentary committee hearings. 
However, this information was prepared for different purposes, at 
different times, and for dissemination through different channels. 


1.75 In the departmental performance reports submitted to 
Parliament in fall 2010, we examined whether departments and 
agencies followed the Secretariat’s guidance on the way to present 
information about progress and spending under the three Economic 
Action Plan programs. 


1.76 We found that Infrastructure Canada’s 2009-10 Departmental 
Performance Report provided information on progress and spending 
under the Infrastructure Stimulus Fund, while Industry Canada’s 
2009-10 Departmental Performance Report provided information 
concerning the Knowledge Infrastructure Program. However, we found 
that, overall, the regional development agencies and Industry Canada’s 
information in departmental performance reports on the Community 
Adjustment Fund was incomplete: the information lacked details on 
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the expected results and did not give a summary of performance to 
date. Consequently, it was difficult to assess program progress. 


177 In its guidance for departmental performance reports, the 
Secretariat asked that departments identify their expected Economic 
Action Plan results, performance indicators, and targets as outlined in 
their reports on plans and priorities for the 2009-10 fiscal year. Several 
officials from the regional development agencies delivering the 
Community Adjustment Fund projects that we audited told us that 
they could not provide this information in their 2009-10 departmental 
performance reports, because their reports on plans and priorities had 
been written before the details of Economic Action Plan programs 
were known. Agency officials indicated that the information on 
program performance would be more comprehensive in departmental 
performance reports for the 2010-11 fiscal year. 


1.78 Following the Secretariat’s guidance, Infrastructure Canada, 
Industry Canada, and the regional development agencies reported 
performance information on their Economic Action Plan programs in 
various places throughout their departmental performance reports. 
According to Secretariat officials, this method of reporting was chosen 
to ensure consistency and alignment with the structure of the 
Estimates documents. In our view, the fragmented presentation made 
it difficult for parliamentarians and Canadians to obtain an overall 
picture of results achieved against planned performance expectations 
and public resources spent. 


1.79 Recommendation. In its future guidance for departmental 
performance reports, the Treasury Board of Canada Secretariat should 
encourage departments and agencies to consolidate Econamic Action 
Plan reporting in a separate sub-section of their departmental 
performance reports. 


The Secretariat’s response. Agreed. The Secretariat released the 
2010-11 guidance for the Departmental Performance Report to federal 
departments and agencies on 18 July 2011. The guidance provides a 
specific request to report Economic Action Plan information, where 
feasible, at the end of Section II of the report. 


Central agencies intend to issue a report on the impact of the Economic Action Plan 


1.80 In our first audit of the Economic Action Plan, we recommended 
that the Department of Finance Canada and the Privy Council Office 
prepare a summary report to Parliament at the conclusion of the 
Economic Action Plan, including a detailed account of the plan’s 


impact on the economy. 
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Delivering time-sensitive programs 


1.81 In their response, the Privy Council Office and the Department 
of Finance Canada committed to supporting the government in its 
reporting on the delivery and economic impact of the Economic 
Action Plan, building on the five reports to Canadians that had been 
provided up to fall 2010. The Department of Finance was monitoring 
the impact of the Economic Action Plan and planned to prepare a final 
report on it. 


1.82 In this audit, the Privy Council Office and the Department 

of Finance Canada informed us that, building on the reports to 
Canadians published to date, the government intends to submit a 
summary report on the delivery and economic impact of the Economic 
Action Plan to Parliament once sufficient information is available, 


probably by late 2011 or early 2012. 


1.83 Department of Finance Canada officials also indicated that a 
summary of Economic Action Plan spending results for the 2010-11 
fiscal year would be included in the Annual Financial Report and 
Public Accounts, to be released in fall 2011. In addition, the 
Department is planning to provide spending results related to the 
extension of selected Economic Action Plan programs in the Annual 
Financial Report and Public Accounts for the 2011-12 fiscal year, 

to be released in fall 2012. 


The experience of delivering Economic Action Plan programs has provided 
important lessons 


1.84 ‘To meet the Economic Action Plan’s objective of stimulating 

the economy quickly, the federal government faced the necessity of 
designing and implementing programs within a limited time frame. In a 
March 2009 letter to the President of the Treasury Board, the Auditor 
General of Canada acknowledged the challenges presented by the 
Economic Action Plan. At the same time, she stressed the importance of 
balancing the government’s wish to move quickly with the requirement 
to give due attention to the spending of a significant sum of public funds. 


1.85 In our first audit of selected Economic Action Plan programs, 
we found that the government had adequately managed the selected 
programs: it put in place appropriate management practices and 
provided programs to eligible recipients in a timely manner. As 
recommended in our first audit, it is important that the federal 
government consider the lessons it has learned and the best practices it 
has developed during the design and implementation of the Economic 
Action Plan, and that it document these for consideration in future 
federal programming. 
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1.86 In both audits, we observed that departments and agencies took 
the initiative to develop new processes that could benefit regular 
federal programming. The following are some of our observations, 
together with the insights gained by departments and agencies from 
the delivery of time-sensitive programs. 


1.87 Lessons learned from the first Economic Action Plan audit 
include the following: 


¢ Review documents for Cabinet committees and submissions for 
Treasury Board financial approval at the same time if there is a 
need to accelerate the design and approval of new programs. 


* To accelerate implementation, deliver new programs through 
existing program and delivery mechanisms. 


1.88 Lessons learned from the second Economic Action Plan audit 
include the following: 


¢ Project approvals require a detailed project schedule. These are 
intended to provide increased assurances that each project will 
meet required deadlines if a program has a compressed time frame. 


¢ For large and complex infrastructure projects, introduce 
additional assurances by engaging professionals (engineers, 
architects) during the approval phase of projects. 


¢ When relying on the provinces and territories to monitor 
contribution agreements, collect timely information to confirm 
that projects are being completed as intended and that committed 
federal funds allocated to projects are spent. 


¢ Ensure that Parliament obtains an overall picture of results 
achieved against planned performance expectations and public 
resources spent. 


Conclusion 


1.89 For the three Economic Action Plan programs we audited, 

we found that the federal government monitored progress and 
spending of projects, and reported performance information on their 
Economic Action Plan programs. 


1.90 Infrastructure Canada, Industry Canada, and the regional 
development agencies collected project information that allowed 
them to take corrective action if projects were falling behind schedule. 
As well, the Infrastructure Stimulus Fund, Knowledge Infrastructure 
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Program, and Community Adjustment Fund largely achieved the 
Economic Action Plan objective to spend federal funding, in a timely 
manner, by 31 March 2011. However, the total federal spending for the 
three programs will be known only when the federal departments and 
agencies receive final project claims and close-out reports. 


1.91 We found that the government's decision to extend the 

31 March 2011 deadline for the Infrastructure Stimulus Fund and 

the Knowledge Infrastructure Program was based on a central agency 
analysis; this considered the information on project progress, the risk of 
stopping federal funding before the Economic Action Plan investments 
could be realized, and the strength of the economic recovery in 
Canada and globally. 


1.92. One of the key objectives of the Community Adjustment 

Fund was to create or maintain jobs in communities hit hard by the 
economic downturn. It remains unclear how the government will 
assess the effectiveness of the Community Adjustment Fund program 
in achieving this objective. 


1.93 Following guidance from the Treasury Board of Canada 
Secretariat, Infrastructure Canada, Industry Canada, and the regional 
development agencies reported performance information on their 
Economic Action Plan programs in various places throughout their 
departmental performance reports. According to Secretariat officials, 
this method of reporting was chosen to ensure consistency and 
alignment with the structure of the Estimates documents. In our view, 
the fragmented presentation made it difficult for parliamentarians and 
Canadians to obtain an overall picture of results achieved against 
planned performance expectations and public resources spent. 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these 
standards as the minimum requirement for our audits, we also draw upon the standards and practices of 
other disciplines. 


Objective 


The objective of our audit was to determine whether selected federal departments and agencies monitored 
and reported on progress and federal spending for selected Economic Action Plan programs. 


Scope and approach 


Our audit focused on the federal government’s monitoring and reporting on progress and spending for 
selected programs of Canada’s Economic Action Plan. We restricted our scope to three programs. The 
scope included two departments and five regional development agencies for detailed audit work: 
Infrastructure Canada and Industry Canada, the Atlantic Canada Opportunities Agency, the Economic 
Development Agency of Canada for the Regions of Quebec, Western Economic Diversification Canada, 
the Federal Economic Development Agency for Southern Ontario, and the Canadian Northern Economic 
Development Agency. The Federal Economic Development Initiative for Northern Ontario is an Industry 
Canada initiative that delivered the Community Adjustment Fund together with the five regional 
development agencies. 


We also looked at the role of the central agencies—the Treasury Board of Canada Secretariat, Department 
of Finance Canada, and the Privy Council Office—in the decision to extend the project deadline. 


Separate statistically representative samples were extracted from the populations of interest, namely 
- the Infrastructure Stimulus Fund, terminating on 31 March 2011; 
« the Infrastructure Stimulus Fund, extended to 31 October 2011; 


- the Knowledge Infrastructure Program, terminating on 31 March 2011; 


. 


the Knowledge Infrastructure Program, extended to 31 October 2011; and 


- the Community Adjustment Fund, terminating on 31 March 2011. 


A total of 236 projects were selected from the combined population of 5,778. Individual sample sizes were 
sufficient in each case to conclude on the respective populations with a confidence level of 90 percent and 
a confidence interval of +10 percent. Eight of the 236 sampled projects were also used as illustrative cases. 


In addition, 8 of the 42 newly approved Infrastructure Stimulus Fund projects and both newly approved 


Knowledge Infrastructure Program projects were selected for our review of project eligibility. 
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Criteria 


progress, including whether projects are being completed as 
intended, for selected Economic Action Plan programs. 


Federal departments and agencies monitor overall project Policy on Management, Resources and Results Structures, 


Treasury Board, 2010 
Policy on Transfer Payments, Treasury Board, 2008 


Directive on Transfer Payments, Treasury Board, 2008 


Central agencies undertake an analysis to inform the decision to 
extend the project completion deadline for selected Economic 
Action Plan programs. 


Privy Council Office, 2010-11 Report on Plans and Priorities, 
“Operational Priority 1” 


Department of Finance Canada, 2010-11 Report on Plans 
and Priorities 


Newly approved projects, as of 20 December 2010, for the 
Infrastructure Stimulus Fund and the Knowledge Infrastructure 
Program meet eligibility criteria, including demonstrating that 
they will meet the 31 March 2011 deadline. 


Federal departments and agencies monitor federal spending, 
including operating costs, for selected Economic Action Plan 
programs. 


Federal departments and agencies report to Parliament on actual 
Economic Action Plan spending, results achieved, and 
performance of their Economic Action Plan programs. 


Program terms and conditions for the Knowledge Infrastructure 
Program, the Infrastructure Stimulus Fund, and the 
Community Adjustment Fund 


Policy on Transfer Payments, Treasury Board, 2008 


Policy on Transfer Payments, Treasury Board, 2008 
Directive on Transfer Payments, Treasury Board, 2008 


Policy on Management, Resources and Results Structures, 
Treasury Board, 2010 


Program terms and conditions for the Knowledge Infrastructure 
Program, the Infrastructure Stimulus Fund, and the 
Community Adjustment Fund 


Policy on Management, Resources and Results Structures, 
Treasury Board, 2010 


Guidance for Preparing the 2009-10 Departmental 
Performance Report, “Canada’s Economic Action Plan,” 
Treasury Board of Canada Secretariat 


Management reviewed and accepted the suitability of the criteria used in the audit. 
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Period covered by the audit 


Audit work for this chapter covers the period from February 2009, when Budget 2009 came into force, 
to 31 March 2011. 


Audit team 


Assistant Auditor General: Ronnie Campbell 
Principal: John Affleck 


Directors: Patricia Bégin, Milan Duvnjak 


Audrey Garneau 
Erin Jellinek 
Jenna Lindley 
Daniel Mukuria 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 


Report of the Auditor General of Canada—Falll 2011 Chapter 1 | 31 


CANADA'S ECONOMIC ACTION PLAN 


Appendix List of recommendations 


The following is a list of recommendations found in Chapter 1. The number in front of the 
recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses 
indicate the paragraphs where the topic is discussed. 


——-Recommendat 
Monitoring progress and spending 


1.71 When federal initiatives to be 
delivered by multiple federal entities are 
developed, the sponsoring departments, 
in consultation with the Treasury Board 
of Canada Secretariat, should ensure 
that programs are designed to allow for 
reliable performance measurement and 
reporting on overall impact and 


effectiveness. (1.67-1.70) 


179 Inits future guidance for 
departmental performance reports, the 
Treasury Board of Canada Secretariat 

_ should encourage departments and 
agencies to consolidate Economic 
Action Plan reporting in a separate sub- 
section of their departmental 
performance reports. (1.72-1.78) 


The Department’s and regional development agencies’ 
response. Agreed. When delivering new national programming, 
Industry Canada and regional development agencies will work 
with federal delivery partners and in consultation with the 
Treasury Board of Canada Secretariat to ensure reliable 
performance measurement and reporting on overall program 
impact and effectiveness. 


The Secretariat’s response. Agreed. As part of its regular 
business and as requested by sponsoring departments, the 
Secretariat will continue to provide advice and guidance on 
performance measurement and reporting in the design of 
programs delivered by multiple departments. 


The Secretariat’s response. Agreed. The Secretariat released 
the 2010-11 guidance for the Departmental Performance 
Report to federal departments and agencies on 18 July 2011. 
The guidance provides a specific request to report Economic 
Action Plan information, where feasible, at the end of Section II 
of the report. 
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Performance audit reports 


This report presents the results of a performance audit conducted by the Office of 
the Auditor General of Canada under the authority of the Auditor General Act. 


A performance audit is an independent, objective, and systematic assessment 

of how well government is managing its activities, responsibilities, and resources. 
Audit topics are selected based on their significance. While the Office may 
comment on policy implementation in a performance audit, it does not comment 
on the merits of a policy. 


Performance audits are planned, performed, and reported in accordance with 
professional auditing standards and Office policies. They are conducted by 
qualified auditors who 


* establish audit objectives and criteria for the assessment of performance; 
* gather the evidence necessary to assess performance against the criteria; 
* report both positive and negative findings; 

* conclude against the established audit objectives; and 


* make recommendations for improvement when there are significant 
differences between criteria and assessed performance. 


Performance audits contribute to a public service that is ethical and effective 
and a government that is accountable to Parliament and Canadians. 


Table of Contents 


Main Points 


Introduction 


Legislative framework governing admissibility 
Our previous audit and subsequent events 
Focus of the audit 


Observations and Recommendations 


Determining admissibility 

Visa officers’ initial training is good and a support network exists 

Visa officers need better tools to help identify potentially inadmissible applicants 
Quality assurance for issuing visas needs to be improved 

Security screening 


The Canada Border Services Agency has not identified all information that may be available 
from its security partners 


The Canada Border Services Agency systems and practices need improvement to ensure 
that sound advice is provided to visa officers 


Medical screening 


A better strategy is needed to protect the health and safety of Canadians 
Better quality assurance is needed to manage and monitor medical risks 


Governance, risk management, and performance measurement 


Citizenship and Immigration Canada and the Canada Border Services Agency continue 
to clarify their relationship 


Joint risk management for admissibility determination needs to be implemented 


Performance measurement information is not available 


Conclusion 
About the Audit 


Appendix 


List of recommendations 
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Main Points 


What we examined = People from other countries—foreign nationals—who want to enter 
Canada as permanent residents must obtain a Canadian visa. Foreign 
nationals who want to enter Canada on a temporary basis must also 
apply for a visa, unless they are from a visa-exempt country. To obtain a 
visa, foreign nationals must meet all requirements for the category 
under which they are applying and must be deemed to be admissible. 
In 2010, 1.36 million visas (including 317,000 permanent resident 
visas) were processed at Canadian missions in foreign countries. 


Admissibility of foreign nationals into Canada falls under the 
Immigration and Refugee Protection Act. The Act defines various 
situations where a foreign national would be inadmissible—for 
example, if the individual presents a risk to the health, safety, or 
security of Canadians. Administering the various provisions of the Act 
is a shared responsibility between Citizenship and Immigration Canada 


(CIC) and the Canada Border Services Agency (CBSA). 


Before issuing a visa, CIC officials must determine that the applicant is 
admissible to Canada. They are supported in making this 
determination by the CBSA, which—with the help of the Canadian 
Security Intelligence Service (CSIS) and the Royal Canadian Mounted 
Police (RCMP)—coordinates and provides intelligence information 


related to the applicants. 


We examined whether Citizenship and Immigration Canada and the 
Canada Border Services Agency have managed the risks associated 
with determining admissibility before issuing a visa, in line with the 
objective of the Act to protect the health, safety, and security of 


Canadians. 


Audit work for this chapter was substantially completed on 
29 April 2011. 
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Why it’s important 


What we found 


Global events in the last decade have changed the nature of threats to 
Canadian society. Diseases prevalent in other countries that can be 
transmitted rapidly worldwide, incidents of terrorism, and organized 
crime around the world have shown the importance of identifying 
individuals who present a risk and preventing their entry into Canada. 


Identifying visa applicants who are inadmissible to Canada is a highly 
complex process that relies heavily on the judgment and experience of 
CIC’s visa officers in missions overseas and on the information made 
available to them. Visa officers are expected to make the best decisions 
they can with that information in the time they have available. It is 
critical that visa officers receive from their security and medical 
partners timely and reliable information on applicants. 


* Citizenship and Immigration Canada and the Canada Border 
Services Agency have taken some measures to address long-standing 
weaknesses in the process of determining whether visa applicants are 
admissible to Canada. However, deficiencies still exist in the measures 
used to identify foreign nationals who may be inadmissible for health, 
safety, or security concerns. CIC and the CBSA lack the necessary 
tools and information to provide assurance that risks related to the 
admissibility determination process are properly managed. 


e 


Some of the tools and risk indicators that visa officers use to identify 
inadmissible persons, and to know when to seek advice from security 
partners, are not kept up to date, nor are they always available. 
Furthermore, many CBSA analysts who provide security advice to visa 
officers have not received the necessary formal training to do so. 
Documentation to support the advice sent to visa officers offered little 
insight into how the analysts made their assessments, and in many 
cases not all the checks that should have been done were completed. 


CIC lacks guidance on the use of two key criteria used in medical 
screening—danger to public health and danger to public safety— 
although it has undertaken some work to explain what they mean. 
Medical screening to determine danger to public health has focused 
mainly on the same two diseases for the past 50 years—syphilis and 
tuberculosis. Although today 56 diseases require national 
surveillance in Canada, CIC has not reviewed whether foreign 
nationals should also be subject to mandatory testing for some of 
these diseases. 
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* CIC and the CBSA do not have systematic quality assurance 
practices or performance measures in place to know how well they 
are identifying individuals who are inadmissible because of health, 
safety, or security concerns. Most quality assurance practices that do 
exist focus on supporting decisions to refuse a visa. Because those 
decisions represent a very small percentage of applications each year, 
this means that the quality of decisions on the vast majority of 
applications is not reviewed. 


The entities have responded. The entities agree with all of our 
recommendations. Their detailed responses follow the 
recommendations throughout the chapter. 
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Visa officers—tEither Canada-based officers 
or locally engaged visa officers. Canada-based 
officers are CIC employees sent abroad from 
Canada who have the designated authority to 
issue visas. Locally engaged visa officers are 
foreign nationals who are employed by a 
mission and have similar responsibilities to 
Canada-based officers, except that they don't 
have the authority to make security-related 
admissibility decisions 
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Introduction 


Legislative framework governing admissibility 


2.1. The Immigration and Refugee Protection Act, which came into 
effect in 2002, governs immigration to Canada. One of its objectives is 
to protect the health and safety of Canadians and maintain the 
security of Canadian society. Citizenship and Immigration Canada 
(CIC) and the Canada Border Services Agency (CBSA) are jointly 
responsible for administering the Act and can deny entry to Canada for 
many reasons, including health, safety, or security concerns. 


2.2. Citizenship and Immigration Canada enables foreign nationals to 
enter Canada as permanent or temporary residents by issuing visas. 
Under the Act, all applicants for permanent residence must obtain a 
visa before they can travel to Canada. A visa is also required for anyone 
applying for temporary residence, except citizens of countries where the 
Canadian government has granted a visa exemption. The temporary 
visa requirement applies to citizens of more than 140 countries. To 
obtain a visa, foreign nationals must meet all conditions for the category 
under which they are applying and must be admissible under the Act. 


2.3 Officials of the Canada Border Services Agency are responsible 
for coordinating, with the Canadian Security Intelligence Service 
(CSIS) and the Royal Canadian Mounted Police (RCMP), information 
related to foreign nationals seeking entry into Canada and providing it 
to CIC officials to use when determining admissibility. Exhibit 2.1 
shows the responsibilities for administering the'key inadmissibility 
provisions of the Act related to health, safety, and security. 


24 In 2010, CIC’s visa officers abroad processed visa applications 
for 1.04 million people seeking temporary residence and for 

317,000 people seeking permanent residence. These officers assess the 
information provided by applicants, collect additional information if 
needed, seek health and security advice from federal partners, and 
make a final decision based on their knowledge, judgment, and use of 
the tools available to them. Exhibit 2.2 shows the process for 


determining admissibility. 


2.5 Visa officers work in a very complex operational environment. 
The type of applications they deal with and the risks to be assessed 
vary from one country to another. In addition, these officers can have a 
heavy workload. Meeting annual immigration targets and having 
service standards of a few days to issue temporary resident visas can 


make the application review process very demanding. 
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Mission—An office of the Government of 
Canada outside Canada. This includes an 
embassy or high commission, consulate 
general, and consulate. 


Exhibit 2.1 Citizenship and Immigration Canada and the Canada Border Services Agency share 
responsibility for the inadmissibility provisions of the /mmigration and Refugee Protection Act 


Responsibility 
for developing 
Section of the Act ce related policies : 
BPS SSS SSUES ER ESS RSS BSS ANE SECS NSS ee NN cea ON BA Sele oN See RMSE 

Section 33: The facts that constitute inadmissibility under 
sections 34 to 37 include facts for which there are reasonable 
grounds to believe that they have occurred, are occurring, or 

may occur. 


CIC and CBSA 


Section 34 (Security): Applicants have engaged in spying, 
subversion, terrorism, or acts of violence, or they belong to 
organizations that have engaged in, or will engage in, these 
activities. 


CBSA 


Section 35 (Human or international rights violations): 
Applicants have committed war crimes or crimes against 
humanity. They are or were senior members or officials of a 
government that has committed acts of terrorism, major 
human rights violations, genocide, war crimes, or crimes 
against humanity. 


CBSA 


Section 36 (1) (Serious criminality): Applicants have been 
convicted of, or have committed, a crime punishable in 
Canada by a maximum term of imprisonment of at least 
10 years. CIC 


Section 36 (2) (Criminality): Applicants have been convicted 
of an offence punishable in Canada or have committed an act 
that is an offence in Canada. 


Section 37 (Organized criminality): Applicants belong to an 
organization that is believed to take part in organized criminal 
activity or to engage in transnational crimes such as people 
smuggling, trafficking in people, or money laundering. 


CBSA 


Section 38 (Health): Applicants’ health conditions are likely 
to be a danger to public health or public safety or may cause CIC 
excessive demands on Canada’s health or social services. 


= 


Note: Other provisions related to inadmissibility, such as misrepresentation and financial reasons, were 
not addressed in the audit. 


Source: Adapted from the /mmigration and Refugee Protection Act 


2.6 Visa officers must also ensure that negative decisions they make 
are well documented, since their work can be subject to appeals 
before the Immigration and Refugee Board or judicial reviews by 

the Federal Court. 


27 During the audit, immigration services abroad were provided 
through 86 Canadian missions where about 270 Canada-based 
officers, 12 medical officers, and 1,305 locally engaged staff—including 
about 160 locally engaged visa officers—worked. 
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Exhibit 2.2 Process for determining admissibility when issuing visas 


Applicant Visa officer 


¢ Completes application ¢ Reviews application and 
applies training, experience, 
and the following tools 
and information: 
— admissibility criteria 
— operational guidance 
— risk indicators 
— country-specific 
contextual information 


Provides supporting 
documentation 
(e.g., police certificate) 


* Interviews applicant 
(in some cases) 


Key partners 


(CBSA, CSIS, RCMP 
CIC Health Branch) 


Seeks expertise/advice 
from key partners 


¢ Provide security screening 
results 


¢ Decides whether applicant 


; at * Provide medical screening 
is admissible under the Act 


results 


Our previous audit and subsequent events 


28  Inour April 2000 Report chapter, The Economic Component of 
the Canadian Immigration Program, we found that visa officers had 
little information and support to determine whether applicants were 
likely to engage in criminal activities or endanger the safety of 
Canadians. In addition, we identified a need for training to improve visa 
officers’ decision-making skills. The audit also found weaknesses in the 
management of the medical admissibility process. We noted the need to 
define danger to public health and public safety and excessive demand 
on health care or social services to ensure that the Department was able 
to comply with the medical inadmissibility provisions of the Act 


a 


need that we had also noted in our 1990 audit. 


2.9 Since our 2000 audit, there have been major changes in the 
delivery of the immigration program. In 2002, the Immigration and 
Refugee Protection Act replaced the Immigration Act of 1976 and 
modified the criteria for admissibility. 


2.10 The Canada Border Services Agency was created in 

December 2003, and responsibilities for intelligence, interdiction, and 
enforcement were transferred to it from CIC. In October 2004, the 
CBSA also became responsible for the immigration program at the 


ports of entry. 
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2.11 In July 2008, CIC and the Public Health Agency of Canada 
(PHAC) signed a memorandum of understanding outlining the 
Department’s responsibilities for health in immigration. The PHAC is 
responsible for providing technical support to the Department on 
national public health issues, while CIC is responsible for policy 
development related to its mandate in immigration. 


2.12 Global events in the last decade have also significantly changed 
the threats to Canadian society. For example, health risks originating 
in other countries can be transmitted rapidly worldwide by travellers, 
and terrorist attacks around the world have heightened Canadians’ 
awareness of security risks. 


Focus of the audit 


2.13 The primary objective of our audit was to determine whether 
Citizenship and Immigration Canada and the Canada Border Services 
Agency have managed the risks associated with determining 
admissibility under the Act’s provisions related to health, safety, and 
security before issuing visas to foreign nationals. We did not look at 
other inadmissibility provisions of the Act, such as misrepresentations 
or financial reasons, nor did we examine exemptions or means to 
overcome inadmissibility. Furthermore, we did not examine other 
strategies used by CIC or the CBSA to deny access to foreign nationals 
who could be a threat to Canada: for example, the interception 
overseas of improperly documented travellers en route to Canada, 
screening at ports of entry, or enforcement activities. 


2.14 Our audit work was conducted primarily at the national 
headquarters of both organizations as well as at CSIS and the RCMP 
We also visited six missions overseas that issued visas. Finally, we 
conducted a survey of all CIC visa officers involved in admissibility 
decisions overseas. 


2.15 The audit focused on the process followed by visa officers 
overseas to make an admissibility decision before issuing a visa. The 
period covered by this audit was mainly from January 2010 to 


April 2011, and the audit work was substantially completed 
on 29 April 2011. 


2.16 More details about the audit objective, scope, approach, and 
criteria can be found in About the Audit at the end of this chapter. 
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Observations and Recommendations 


2.17. Citizenship and Immigration Canada (CIC) visa officers are 
responsible for determining the admissibility of applicants to Canada 
before issuing visas. This process requires an understanding of the 
Immigration and Refugee Protection Act, which governs immigration to 
Canada. The admissibility determination process relies heavily on 
information provided by the applicant, who is legally obligated to 
respond honestly to questions on the application form. Some 
applicants, however, may provide false information or fraudulent 
documentation. The admissibility determination process, therefore, 
relies on visa officers’ judgment, training, and experience to analyze 
and validate information they receive from applicants. Visa officers 
also take into consideration the information provided by key partners. 
They are expected to make the best decisions they can with the 
information they have available. According to CIC data, of the 
applications processed in 2010 for 317,000 people seeking permanent 
residence, close to 69,000 applicants were refused, including 817 for 
health, safety, or security concerns. In 2010, CIC also processed 
applications for close to 1.04 million people seeking temporary 
residence and rejected about 189,000 applicants. At the time of our 
audit, information indicating which of these cases were rejected for 
health, safety, or security concerns was not available. 


2.18 Given the challenging context in which visa officers operate, 
they need to be supported with adequate training and tools to do their 
job effectively, particularly in situations where'the visa officer has not 
yet had job experience to draw upon. 


Visa officers’ initial training is good and a support network exists 


2.19 At the time of our audit, about half of the Canada-based officers 
posted abroad and responsible for determining admissibility had less 
than five years of experience in issuing visas. We examined whether 
CIC developed and implemented training to ensure that its officers 


have the competencies to do their jobs. 


2.20 We found that the initial mandatory training on the Act offered 
to officers at the beginning of their careers is well structured and 
rigorous. In addition, refresher training is mandatory for officers who 
are returning to overseas positions after being in Canada in positions 


that did not involve processing visa applications. 
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2.21 Once posted at missions overseas, visa officers are expected to 
maintain their knowledge and keep up to date through self training, 
consultation with peers, and experience gained while processing files. 
They may also receive formal and informal training overseas, although 
we found that this training is offered on an ad hoc basis and varies 
across missions. The annual completion of appraisals and learning 
plans can help identify training needs. However, there is no assurance 
that all visa officers receive the additional training they need to 
maintain and build on their knowledge. In their responses to a survey 
we conducted as part of our audit, visa officers indicated that the top 
sources of additional training in admissibility decision making were 
self-training (87 percent) and coaching by supervisors or colleagues at 
the mission (65 percent). 


2.22 Visa officers also have access to a support network of colleagues 
and supervisors within the mission and can consult officials in Ottawa 
if necessary. In our survey, visa officers stated that they use the network 
often. 


Visa officers need better tools to help identify potentially inadmissible applicants 


2.23 We examined whether the tools visa officers need were 
developed and updated and whether the officers were informed of any 
updates and changes. We looked at 


* operational guidance, 
* screening manuals and risk indicators, and 
* country-specific risk profiles. 


2.24 Operational guidance, risk indicators, and country-specific risk 
profiles are the primary tools to inform visa officers of what risks to 
look for; therefore, these tools need to be kept current so that they are 
useful for identifying people who may be inadmissible. See our overall 
recommendation at paragraph 2.36. 


2.25 Operational guidance. We noted that operational guides and 
bulletins on procedures for applying the Act were available 
electronically to visa officers who, for the most part, found them useful 
and sufficient. However, the limited search capability within the 
hundreds of various manuals and bulletins hampered officers’ ability to 
find the right information quickly or to ensure that they were 
complying with procedures, especially if officers were inexperienced. 
We also found that guidance was not always developed in a timely 
manner to reflect the impact that some court decisions have had on 
the processing of applications. 
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Security concerns—Risks of espionage, 2.26 Screening manuals and risk indicators. Canada-based officers 

terrorism, and subversion (national security ; ndedeitiah diff er ee ee eee | 

seh we tallingll in section $4 of the Acti: are provided with three different screening manuals that define the 

human or international rights violations risk indicators to help them identify applicants who may be 

(section 35); and organized crime (section 37) : wee p : , ; 
inadmissible for security concerns and to determine when to refer a 

file to security partners (the Canada Border Services Agency, the 

Canadian Security Intelligence Service, and the Royal Canadian 

Mounted Police) for advice. In their responses to our survey, 

97 percent of Canada-based officers indicated that they rely on these 


manuals to assess risks. 


2.21 The Canada Border Services Agency (CBSA) is responsible for 
updating risk indicators, but we found that two of the screening 
manuals had not been updated for several years; one of these was last 
updated in 1999. The third manual, which focuses on national 
security, had been updated more frequently and was going through 
another review at the time of our audit. However, despite the CBSA’s 
responsibility under the Act for the inadmissibility provisions related to 
security concerns, the Agency was not directly involved in the review, 
which could lead to inconsistency between the three manuals. 


2.28 We noted that very few applicants, referred by visa officers using 
the current risk indicators, were found to be likely inadmissible by - 
security partners. In many cases, there may be no information or 
concerns related to applicants. Of the cases security partners worked 
on in 2010, only about | percent of applicants for temporary residence 
and 0.1 percent of applicants for permanent residence were found to 
be likely inadmissible (Exhibit 2.3). We noted that there has been no 
analysis to determine whether the current riskindicators to help 
identify potentially inadmissible applicants are appropriate or properly 
applied. 


Exhibit 2.3 Only a small number of applicants who were referred to security partners were identified 
as likely inadmissible in 2010 


Applicants identified as likely 
inadmissible 


_ Cases finalized by 
Security partners 


Temporary residence 


946,000 74,000 622 
Permanent residence 
257,000 42,000 46 


*Applicants subject to criminal and security checks include those 18 and over, or someone younger if 
concerns exist 
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2.29 Country-specific risk profiles. We also found that, upon visa 
officers’ arrival at a new mission, information on country-specific risks 
is not systematically provided to them. For example, during our site 
visits, we noted that some missions provided newly posted visa officers 
with important contextual information specific to the country. At 
other missions, officers commented that a lack of country-specific 
information was affecting their ability to make admissibility decisions. 


2.30 The CBSA produces country assessment reports that provide 
another source of information to visa officers to help them identify 
risks. We found that these reports are not systematically produced and 
distributed to visa officers at missions abroad. They are produced based 
on emerging events or on an ad hoc basis to address a specific request 


by CIC and CBSA officials. 


2.31 Measures for validating information from applicants. The 
reliability of information provided by applicants can be validated 
through measures such as 


* interviewing the applicant, 
* verifying documents with the issuing authorities, 


* seeking further documentation and clarification from applicants, 
or 


* conducting site visits. 


2.32 These measures, however, require time and resources. For 
example, visa officers at missions we visited told us that face-to-face 
interviews were rarely conducted. Since CIC does not systematically 
track when interviews are conducted, data on interview frequency was 
not available. At the end of our audit, CIC was implementing a case 
management system that could allow it to collect interview data. 


2.33 In response to our survey, 65 percent of visa officers indicated 
that the inability to validate applicant information was a challenge in 
determining admissibility. About half of the Canada-based officers 
indicated that they often did not have sufficient information from 
applicants to assess whether an applicant was inadmissible due to 
security concerns. 


2.34 Similarly, assessing inadmissibility due to criminality concerns 
can be challenging. For example, applicants who intend to stay in 
Canada for a minimum of six months must submit a police certificate 
from an official issuing authority of each country where they have 
resided for longer than six months in the last ten years. We were told 
by visa officers at missions we visited that the authenticity of police 
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certificates was often difficult to validate, depending on the country of 
origin. Unless visa officers can confirm the information with reliable 
issuing authorities, few other mechanisms exist to verify that the 
applicant has not been convicted of any crimes. 


2.35 At the end of our audit, CIC had developed and approved a 
Program Integrity Framework that highlights the need to conduct 
antifraud activities, such as verification of applicant information. CIC 
plans to phase in this framework over the next five years. 


2.36 Recommendation. The Canada Border Services Agency and 
Citizenship and Immigration Canada should ensure that operational 
manuals, risk indicators, and relevant country-specific information are 
complete, up to date, and made available to visa officers in a timely 
manner to help them identify foreign nationals who may be 
inadmissible. 


The entities’ response. Agreed. A review of current risk indicators has 
begun in consultation with security partners. Also in consultation with 
security partners, a strategy will be developed to ensure that necessary 
country-specific information is available in a timely manner to support 
admissibility decision making. In addition, a review and update of 
policy manuals will be undertaken to ensure that accurate operational 
guidance is in place for all staff and management involved in the 
admissibility screening process. These measures will be completed by 


June 2012. 


ee 


Quality assurance for issuing visas needs to be improved 


2.37 Given the complexity of the process for issuing visas, we 
examined whether progress has been made in implementing a quality 
assurance framework to address recommendations we made in 
previous audits. Structured quality assurance practices can provide 
assurance on the overall quality of work performed and on how well 
risks are managed. For example, they can show whether risk indicators 
are applied as intended, screening is identifying inadmissible persons, 
and improvements are made as needed. 


2.38 We found that CIC had not yet implemented the quality 
assurance framework it had developed in response to our 2000 audit. 
We noted that some missions undertake their own quality assurance 
activities to review decision making, depending on local priorities and 
the availability of resources. However, such activities are still not 


mandatory and are not performed systematically across missions. 
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Security screening 


Security screening—Procedures used to 
identify foreign nationals who may be 
inadmissible because they are, have been, or 
are likely to be involved in activities related to 
espionage, terrorism, and subversion; human 
or international rights violations; or organized 
crime. 


2.39 The Department’s 2011 Program Integrity Framework calls for 
the monitoring of the quality of decision making through random, 
systematic, and targeted quality assurance activities. We encourage 
CIC to implement this new framework in accordance with its approved 
plan and its commitment made in response to a recommendation in 
our November 2009 Report chapter, Selecting Foreign Workers Under 
the Immigration Program, to apply quality assurance in a consistent 
and risk-based fashion within two years. 


2.40 Applicants aged 18 years and over are subject to criminal and 
security checks. (A visa officer can also request checks on someone 
younger if concerns exist.) First, visa officers examine the information 
and documentation submitted by the applicant. If they determine that 
an applicant requires a more detailed investigation, the applicant’s file 
is referred to partners for security screening. 


2.41 The Canada Border Services Agency (CBSA) is responsible for 


supporting the security screening of persons who may be inadmissible. 
The Agency is the central liaison for the security screening process in 
partnership with the Canadian Security Intelligence Service (CSIS) 
and the Royal Canadian Mounted Police (RCMP). The results of the 
security screening are key to visa officers when making a final 
determination of an applicant’s admissibility. 


The Canada Border Services Agency has not identified all information that may be 
available from its security partners 


2.42 We examined whether Citizenship and Immigration Canada 
(CIC) and the CBSA have identified what information is available and 
useful, from the RCMP and CSIS, and whether they obtain it in a 
timely manner. Memoranda of understanding between CIC and its 
security partners state that relevant information held by partners will 
be provided on a timely basis. | 


2.43 We found that neither CIC nor the CBSA has conducted formal 
assessments to ensure that they have a thorough understanding of the 
nature and type of information that security partners could provide 
and that all key information needed to support security advice and 
recommendations is made available. 


2.44 CSIS is the primary partner for security screening; it is involved 
in all screening requests related to national security. The RCMP has 
very limited involvement in the screening process. While the CBSA 
has access to some criminal databases, it does not systematically 
consult the RCMP to obtain intelligence on organized crime concerns. 
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2.45 We also found that since its creation, the CBSA has not signed 
memoranda of understanding or agreements on service levels with 
either of its security partners. Agreements related to obtaining 
information required for security screening date back to 2002 between 
CIC and CSIS and between CIC and the RCMP before CBSA was 
created and given responsibility for security screening. During our 
audit, three of the organizations—the CBSA, CSIS, and CIC— 
initiated a review of their sharing of information for the purposes of 
screening foreign nationals. 


2.46 The results of security screening depend on the information 
available to security partners and, in almost all cases, they have little or 
no evidence to suggest that foreign nationals seeking entry to Canada 
are inadmissible. However, where there is information, it is important 
that it be made available to visa officers in a usable and timely manner. 


2.41 Recommendation. The Canada Border Services Agency should 
ensure that all information that can be obtained from security partners 
and is relevant to security screening is used to provide advice to 
Citizenship and Immigration Canada. 


The Agency’s response. Agreed. A comprehensive approach will be 
developed to ensure that all relevant information is available and 
shared with Citizenship and Immigration Canada (CIC). An 
assessment of information needs will take place in consultation with 
security partners, identifying any gaps and resulting in an information 
management strategy to ensure that necessary information can be 
accessed to support admissibility decision making. In addition, existing 
information-sharing arrangements between CIC, the Canada Border 


Services Agency, the RCMP and the Canadian Security Intelligence 
Service will be reviewed to ensure that they are complete and serve the 
needs of the security screening program. These measures will be 


completed by March 2013. 


The Canada Border Services Agency systems and practices need improvement to 
ensure that sound advice is provided to visa officers 


2.48 The memorandum of understanding between CIC and 

the CBSA highlights the need to provide, in a timely fashion, 
necessary information to staff to perform their duties. We examined 
whether the CBSA has systems and practices in place to provide 
information and advice that support the decision-making needs of visa 
officers. We reviewed a representative sample of CBSA files finalized in 
2010—109 temporary residence files and 50 permanent residence files. 


See our overall recommendation at paragraph 2.63. 
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2.49 Quality assurance and effectiveness. We found that the CBSA 
does not have a systematic quality assurance process to verify the 
quality and consistency of the reviews conducted by its analysts. The 
Agency’s officials indicated that senior analysts review all negative 
briefs (inadmissibility recommendations) before sending them to the 
missions and that complex cases are often discussed with senior 
analysts or with management. However, negative recommendations 
represent less than one percent of the temporary residence workload 
and eight percent of the CBSA’s permanent residence files. In our 
view, since the risk is that someone will be admitted to Canada who 
should not be, it is important to do quality assurance on all files and 
not just on the few where analysts make negative recommendations. 


2.50 We also noted that the CBSA has not reviewed the effectiveness 
of its security screening process. It has not requested feedback from 
CIC on the usefulness of the information provided to visa officers, and 
there is no process to find out how they use the information. In our 
survey, about 45 percent of the Canada-based officers indicated that 
one of the challenges in determining the inadmissibility of an applicant 
is the lack of relevant information from security partners. 


2.51 Timeliness of security screening. We also examined whether 
the results of the CBSA’s security screening provided to Canada-based 
officers are timely. We found that, for the most part, the Agency 
responded in a timely manner for temporary residence cases. 


2.52 The temporary residence screening manual indicates that the 
CBSA will reply to a visa officer’s referral within 10 working days. Our 
file review showed that the CBSA responded to the missions within 
10 working days in 80 percent of the cases. However, we found that in 
at least 25 percent of those cases, it had not waited for CSIS input 
before providing the security screening results. Therefore, the results 
provided to Canada-based officers did not include the CSIS review. 
This occurred because the CBSA’s data system automatically sends a | 
response to the missions after 10 days, unless an analyst has put a hold 
on the file. We were told, however, that if a concern is identified by 
CSIS after 10 days, the CBSA would notify the mission to stop the visa 
issuance. Nevertheless, the current practice allows for the exclusion of 
potentially important information due to the possibility that CSIS is 
unable to provide a response within 10 working days. 


2.53 For permanent residence cases, we noted that there was a formal 
agreement on service standards in the 2006 memorandum of 
understanding between CIC and the CBSA, which stated that advice 
will be provided to visa officers within 9 to 18 months. Through our 
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file review, we found that the CBSA had completed its analysis in less 
than 9 months in 67 percent of the cases and exceeded 18 months in 
19 percent of the cases. 


2.54 We also reviewed a sample of 10 percent of the 72 permanent 
residence cases analyzed by the CBSA after CSIS had completed its 
review for national security concerns. In the cases we examined, we 
found that CSIS took an average of 34 months to provide its results to 
the CBSA when there was information requiring further investigation. 
We noted that the CBSA concurred with the advice provided by CSIS 
but, on average, it took an additional 9 months to send the advice to 
the visa officer without adding specific details about the individual. 


2.55 In responding to our survey, 46 percent of the Canada-based 
officers indicated they were concerned with the length of time it can 
take to obtain information or advice from security partners. The CBSA 
has recognized that timeliness is a concern and, at the end of our audit, 
had initiated measures to improve the timeliness of its screening. 


2.56 IT systems used in security screening. The CBSA relies on its 
Secure Tracking System, an automated system that processes and 
manages temporary residence cases. However, the system does not 
search through all information available from the Agency, whether in 
its intelligence products or in other databases. 


2.57 Analysts can consult various other databanks. However, the 
Agency identified some concerns about these databanks in a 2011 
report on the security screening unit. The report identified the 


ov 


following IT system risks: 


* The systems were not interoperable with those of security 
partners, which created barriers to obtaining necessary 
information in a timely manner. 


¢ Staff lacked complete access to all relevant CBSA systems, which 


created information gaps resulting in recommendations made by 


staff based on incomplete information. 


* The systems were old and needed to be replaced; the necessary 
upgrades and changes to modernize the systems were not a 


priority. 


The report also indicated that actions were under way to mitigate the 


risks. We did not examine the implementation of these actions. 


258 Training and support for Agency analysts. We examined the 
training provided to CBSA analysts to ensure that they have the 


necessary knowledge and skills to perform their duties. We found that 
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there was no formal training curriculum developed specifically for 
analysts based on required competencies, and that the Agency had 
already identified gaps in the training that was provided. For example, 
at the time of our audit, 39 percent of analysts had not received 
training in the provisions and requirements of the Immigration and 
Refugee Protection Act, and 74 percent were missing training in research 
techniques. 


2.59 Analysts told us that in the absence of a formal training program, 
they rely mostly on guidance material, coaching, and on-the-job 
training to acquire the knowledge they need to fulfill their 
responsibilities. We noted that more than 40 percent of staff had 

two years of experience or less and that there was little stability at the 
senior levels to provide coaching and on-the-job training. The lack of 
training combined with staff turnover has contributed to an 
environment that cannot ensure that analysts are providing the best 
information or advice to visa officers. 


2.60 Our file review indicated that CBSA analysts did not 
consistently document the work they performed to provide advice on 
the admissibility of temporary residence applicants. We found such 
documentation in only 28 percent of the files reviewed. In those cases, 
analysts had listed the databases they consulted and indicated whether 
an Internet search was done but had not described the review 
performed or the information found. As a result, the Agency could not 
provide assurance that analysts had followed procedures or had 
conducted a full assessment. 


2.61 In our review of permanent residence applicant files, we found 
that they were better documented and the reviews completed were 
more comprehensive than those for temporary residence. We were 
able to determine the extent of the review conducted by analysts 

in 70 percent of the files. Of these files, however, we found that all 
the mandatory checks were not completed in more than 80 percent 
of cases. CBSA analysts did not consult the required databases and 
Open sources as per CBSA guidelines. Furthermore, they did not use 
the checklists that had been designed to guide them in their review 
to provide some assurance of completeness. 


2.62 At the end of the audit, CBSA senior management confirmed 
training as a priority and indicated that training needs were being 
identified for security screening analysts. 
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Medical examination—A mandatory 
assessment for all applicants for permanent 
residence and their family members, whether 
or not they will accompany the applicant to 
Canada. A medical examination is required for 
some temporary residence applicants 
depending on their country of residence, length 
of stay, and planned activities in Canada— 
for example, working in an occupation in which 
the protection of public health is essential. 
Visa officers can also request a medical 
examination if they deem it necessary in 

other circumstances. 
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2.63 Recommendation. The Canada Border Services Agency should 


* implement a quality assurance process to ensure the consistency 
and quality of information and advice provided by analysts to 
Citizenship and Immigration Canada (CIC); 


* ensure that analysts have adequate training, support, and tools to 
perform their duties; 


¢ establish service standards for the timeliness of security screening 
based on the needs of CIC and its own capacity and that of its 
security partners; and 


* measure the effectiveness of its security screening to ensure that it 
meets the needs of CIC in a timely manner. 


The Agency’s response. Agreed. The Canada Border Services Agency 
(CBSA) has restructured its security screening program to provide more 
focus on quality assurance and performance management. A review of 
training requirements for screening officers is under way and a high-level 
plan has been adopted to provide screening officers with a competency- 
based training program and supporting tools, to be implemented 
incrementally. In consultation with Citizenship and Immigration Canada 
and security partners, the CBSA will review service standards and make 
necessary program adjustments to ensure that the needs of all partners 
are met. These measures will be completed by December 2012. 


2.64 Under the Immigration and Refugee Protection Act, foreign 
nationals are inadmissible for reasons of health“if their health condition 


* is likely to be a danger to public health, 
* is likely to be a danger to public safety, or 


* might reasonably be expected to cause excessive demand on 
Canada’s health or social services. 


2.65 In order to assess an individual’s health status, Citizenship and 
Immigration Canada (CIC) has 15 medical officers working in 

10 regional medical offices worldwide. They oversee a network that 
includes laboratories, clinics, and more than 1,000 local physicians 
designated by CIC in 189 countries. Health infrastructure varies 
greatly from country to country, which adds to the challenge. These 
medical officers are responsible for ensuring that appropriate medical 
examinations are conducted and that visa officers are provided with 
any necessary information before making an admissibility 
determination. 
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A better strategy is needed to protect the health and safety of Canadians 


2.66 We examined the systems and practices in place to provide visa 
officers with timely and reliable information to assess if applicants are 
medically admissible. See our overall recommendation at 

paragraph 2.77. 


2.67 Danger to public health and danger to public safety. In order 
to protect Canadians, there must first be a clear understanding of what 
constitutes a danger to public health and a danger to public safety to 
ensure that appropriate medical tests are conducted for foreign 
nationals applying to come to Canada. Although CIC has initiated 
work on what danger to public health and danger to public safety mean 
for immigration purposes, we found that the Department has not yet 
defined these two key provisions of the Act—a concern we also raised 


in our 1990 and 2000 audit reports. 


2.68 At the time of our audit, the Department had defined only 

two diseases—syphilis and tuberculosis—as dangers to public health. 
These same two diseases have defined the screening practice for the last 
50 years. We noted that mandatory HIV testing has been implemented 
since 2002, with the anticipated public health benefit of early detection, 
treatment, and prevention. Persons with HIV, however, will not be 
denied access to Canada for public health reasons. They would be 
found inadmissible only if their health condition might reasonably be 
expected to cause excessive demand on health or social services. 


2.69 We also noted that the Department has not developed an overall 
strategy, based on risks, to screen for danger to public health. The 
Public Health Agency of Canada has identified 56 diseases that require 
national surveillance in Canada, but CIC has not conducted a review 
to determine whether foreign nationals should be subject to mandatory 
testing for some of these diseases in order to protect public health. 


210 Some guidance exists on what medical conditions may constitute 
a danger for public safety. CIC has been focusing its screening on 
psychiatric conditions and drug/alcohol addiction. However, the 
Department’s medical officers lack guidance to assess whether a person 
is likely inadmissible under this provision of the Act. 


2.11 Excessive demand on health or social services. The 
Immigration and Refugee Protection Regulations define excessive demand 
on health or social services as a situation where 


* anticipated costs would likely exceed the average per capita cost 
of health or social services in Canada over a five- or ten-year 
period, or 
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¢ the demand would add to existing waiting lists and increase the 
rate of mortality and morbidity of Canadians. 


2.12 We examined the systems and practices in place to provide visa 
officers with timely and reliable information on whether an applicant 
might cause excessive demand on health or social services. CIC 
medical officers are required to assess the anticipated costs of providing 
health care and social services related to the health conditions of 
applicants. We found, however, that there are limits to their ability to 
accurately estimate these costs because information on provincial/ 
territorial health care expenditures and wait times in Canada for the 
numerous health conditions that exist may not be available to them. 


2.13 Furthermore, because of several court decisions, visa officers 
must also now consider non-medical factors, such as an applicant’s 
ability and willingness to pay for services, or ability to obtain insurance. 
Visa officers explained to us that it is very difficult to assess the intent 
or capacity of an individual to pay. Concerns also exist about the 
unenforceable nature of an applicant’s commitments once the person 
is residing in Canada. 


2.74 Performance management. We found that, in 2010, CIC 
conducted more than 545,000 medical examinations resulting in some 
1,200 applicants (0.22 percent) being found inadmissible for health 
reasons. Of those, less than 2 percent were considered a danger to 
public health or safety (the others were denied due to excessive 
demand on health or social services). CIC officials explained that 
persons who initially might not be admissible for public health reasons 
are able to enter Canada once they have been treated: for example, 

if they can demonstrate that their tuberculosis is inactive. 


215 In our survey, visa officers indicated that they are provided with 
timely advice on medical admissibility. We found that CIC monitors 
the processing times only in those cases where applicants are 
admissible, and not in more complex cases, where applicants have 
conditions that could make them inadmissible. 


2.76 At the time of the audit, CIC was in the process of implementing 
a new performance management framework to measure, monitor, and 
report on the efficiency and effectiveness of its health screening 


process. 


2.71 Recommendation. In order to meet the objectives of the 
Immigration and Refugee Protection Act to protect the health and safety 
of Canadians, Citizenship and Immigration Canada should 
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¢ develop a strategy based on risks to better identify applicants who 
present a danger to public health or a danger to public safety, and 


* examine the methodology and process for assessing excessive 
demand on health and social services and take corrective 
measures as necessary. 


The Department’s response. Agreed. Citizenship and Immigration 
Canada (CIC) has initiated and continues to develop a risk assessment 
and management strategy to better address danger to public health and 
danger to public safety in the immigration context. Among the 
activities conducted are the completion of a discussion paper on 
defining danger to public health, and the completion of an HIV policy” 
review. CIC will implement a strategy that will result in consistent 
admissibility criteria, standardized processes, and improved monitoring 
of its programs. These measures will be completed by September 2013. 


~ ClC has started examining the excessive demand processes and will 


pursue its collaboration with the provinces and territories to review 
factors generating limitations and inconsistencies in the evaluation of 
excessive demands and will address the identified deficiencies. This 
measure will be completed by June 2013. 


Better quality assurance is needed to manage and monitor medical risks 


2.78 In order to ensure the reliability of the medical examination 
process and to mitigate the risk to program integrity, a robust quality 
assurance framework is needed to monitor the work of CIC-designated 
local physicians, laboratories, and X-ray clinics and provide 
information on results. We examined the quality assurance 
mechanisms for the medical reports prepared by those physicians and 
the procedures to monitor the effectiveness of medical screening. 


2.79 We found that activities to assess the quality and reliability of the 
medical examinations performed by CIC- designated local physicians 
varied from one regional medical office to another, and a standard 
approach based on risks has not yet been developed. 


2.80 We also noted that CIC had no standard approach for ensuring 
the quality or consistency of the work done by medical officers or by 
their staff when assessing medical admissibility. At the time of the 
audit, CIC was planning to implement an electronic medical system to 
improve the efficiency and effectiveness of medical assessments and to 
build in quality assurance mechanisms. The Department had also 
recently identified best practices in its regional medical offices to 
develop a standard quality assurance process. 
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2.81 Recommendation. Citizenship and Immigration Canada should 
implement a standard quality assurance process to protect the integrity 
of the medical examination system and to ensure consistency and 
quality in the assessment of medical admissibility. 


The Department’s response. Agreed. Citizenship and Immigration 
Canada (CIC) has already completed the development of a Quality 
Assurance Framework for its immigration medical examination and 
assessment programs. This framework will be integrated into the 
departmental Program Integrity Framework, providing the tool 
required to monitor and evaluate the quality of its immigration health 
program, worldwide, including the designated medical practitioners, 
other examiners, and CIC employees involved in the health 
assessment process. CIC is also in the process of implementing 
eMedical (an electronic medical system), enabling standardization and 
centralization of the medical examination process. These measures will 


be completed by March 2013. 


2.82 The Immigration and Refugee Protection Act is highly complex, 
addressing all aspects of the immigration program. While Citizenship 
and Immigration Canada (CIC) and the Canada Border Services 
Agency (CBSA) are each responsible for developing policies for 
different sections of the Act, they share responsibility for the overall 
delivery of the immigration program. See our overall recommendation 
at paragraph 2.94. 


Citizenship and Immigration Canada and the Canada Border Services Agency 
continue to clarify their relationship 


2.83 Given their shared responsibility for achieving the Act’s 
objectives, we examined whether the CBSA and CIC have defined 
their roles, responsibilities, and expectations regarding the admissibility 
provisions of the Act. 


2.84 After immigration functions previously under CIC’s 
responsibility were transferred to the CBSA, both organizations signed 
a memorandum of understanding (MOU) in 2006 to specify their 
respective roles and responsibilities. The Agency would provide 
guidance and support to the Department, and visa officers would be 
responsible for the final decisions on admissibility. 

2.85 Since 2006, various reviews and senior-level joint committees 
have identified areas where the working relationship between 

the two organizations could be further clarified and strengthened. 

A 2010 joint internal audit on the implementation of the 2006 MOU 


Report of the Auditor General of Canada—Fall 20 11 


Chapter2 | = 23 


ISSUING VISAS 


recommended improvements to governance and, in May 2011, a 
revised MOU and a governance annex were signed. Other annexes 
related to information management and shared services were still 
under negotiation at the end of our audit. 


Joint risk management for admissibility determination needs to be implemented 


2.86 In their policies, both CIC and the CBSA have recognized that 
risk management is an essential element of public sector operations 
and provides a systematic approach to the identification, analysis, 
assessment, treatment, and ongoing monitoring of risks. 


2.87 In their response to a 2010 joint internal.audit, CIC and CBSA. 
agreed to develop an integrated approach to manage the risks related 
to the immigration program. We examined whether the 

two organizations have developed and implemented a joint risk 
management strategy for identifying risks and mitigation strategies 
related to applying the inadmissibility provisions of the Act. 


2.88 We found that CIC and the CBSA have begun to develop a joint 
risk management strategy for the shared delivery of the immigration 
program. Although this strategy was not complete at the end of our 
audit, the Department and the Agency were working on a risk-based 
strategy for setting joint intelligence and enforcement priorities. 


2.89 We are concerned, however, that joint risk management has not 
been implemented. CIC and CBSA had already identified in 2006 that 
quality assurance practices, the routine use of screening tools, and 
applicant interviews were needed as risk mitigation measures. This 
audit found deficiencies in each of these measures. 


Performance measurement information is not available 


2.90 As part of the process to develop a joint risk management 
strategy, CIC and the CBSA were also developing a joint performance 
measurement framework. Together, these initiatives were intended to 
provide officials with a means of understanding how well the 

two organizations were working together in the shared delivery of the 
immigration program. 


2.91 We did note that progress had been made on developing a 
performance measurement strategy, but challenges remain in obtaining 
reliable performance information on security screening. For example, 
CBSA officials informed us that their information system was not 
designed to provide processing times or to identify the number of cases 
reviewed by an analyst versus those finalized by the automated system. 
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2.92 As well, the lack of quality assurance throughout the 
admissibility process has prevented the Department and the Agency 
from obtaining data to identify strengths and weaknesses in their 
systems, or obtaining information on how well their systems work to 
identify people who may be inadmissible for health, safety, and security 
concerns. 


2.93 The two organizations have not measured the overall 
effectiveness of the inadmissibility provisions of the Act. However, 
during our audit, CIC and the CBSA were conducting a review of 
Canada’s admissibility policy to assess whether it still meets Canadian 
needs, to identify any gaps, and to recommend amendments to 
relevant legislation, regulations, or administrative guidelines. 


2.94 Recommendation. Citizenship and Immigration Canada and 
the Canada Border Services Agency should fully implement their joint 
risk management and performance measurement strategies and 
monitor the results. 


The entities’ response. Agreed. Citizenship and Immigration Canada 
(CIC) and the Canada Border Services Agency (CBSA) will 
implement and monitor their joint risk management and performance 
strategies for the shared delivery of the immigration and refugee 
program, including visa issuance. In this context, CIC and the CBSA 
will complete a review of Canada’s admissibility policy, in collaboration 
with security partners, in order to assess whether it is continuing to 
meet the needs of Canada and Canadians; to identify any gaps; and to 
recommend amendments to relevant legislation, regulations, or 
administrative guidelines. 


CIC and the CBSA will complete a joint priority-setting exercise based 
on key risks with respect to the intelligence program (including visa 
issuance) and resulting enforcement priorities. CIC and the CBSA will 
also complete work on key performance indicators with respect to visa 
issuance, which will be used to monitor the effectiveness of shared 


programs and to track the implementation of joint priorities. 


These measures will be completed by December 2012. 


Conclusion 


2.95 Citizenship and Immigration Canada (CIC) and the Canada 
Border Services Agency (CBSA) have taken some measures to address 
long-standing weaknesses in the process of determining whether visa 


applicants are admissible to Canada. In our opinion, however, they 
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have not fully managed the risks associated with determining 
inadmissibility under the Immigration and Refugee Protection Act for 
health, safety, and security concerns before issuing visas to foreign 
nationals. 


2.96 CIC and the CBSA do not have systematic mechanisms for 
quality assurance or measuring performance that would provide a 
reasonable level of assurance that their processes are working and that 
practices are appropriate for today’s challenges. Furthermore, the 
organizations have only recently begun to develop a joint risk 
management approach for their shared delivery of the immigration 
program. 


2.91 ‘Tools and guidance available to visa officers are not always kept 
up to date. The screening for danger to public health is still primarily 
based on the same diseases that have been the focus for the last 

50 years. In addition, many risk indicators, key to identifying 
potentially inadmissible foreign nationals, have not been reviewed or 
updated for years. 


2.98 Similar issues have been identified in our audits since 2000. 
There needs to be a sustained effort by CIC and the CBSA to address 
the gaps in the admissibility determination process so that the related 
risks are properly managed. 
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About the Audit 


All of the audit work in this chapter was Conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these 
standards as the minimum requirement for our audits, we also draw upon the standards and practices of 
other disciplines. 


Objective 


The audit objective was to determine whether Citizenship and Immigration Canada (CIC) and the 
Canada Border Services Agency (CBSA) have managed the risks associated with determining 
admissibility under the Immigration and Refugee Protection Act for health, safety, and security concerns 
before issuing visas to foreign nationals. 


Scope and approach 


The audit examined how the Act’s objective of “protecting the health and safety of Canadians and 
maintaining the security of Canadian society” was being jointly administered by the two primary entities 
involved in admissibility decisions—CIC and the CBSA. Our audit covered the process followed by visa 
officers to make the admissibility decision before issuing visas to foreign nationals seeking entry into 
Canada as either temporary or permanent residents. We did not look at other inadmissibility provisions of 
the Act, such as misrepresentations or financial reasons, nor did we examine exemptions or means to 
overcome inadmissibility. 


The issuance of visas is one component of Canada’s strategy to deny access to foreign nationals who may 
present risks to Canadians. Other components include, for example, the interception overseas of 
improperly documented travellers en route to Canada, screening at ports of entry, and enforcement 
activities, including investigation, detention, and removals. Our audit did not examine these activities. 


At CIC, we interviewed officials at headquarters and at overseas missions involved in processing 
applications. We held focus groups with visa officers responsible for making admissibility decisions. 

We reviewed files and reports, and analyzed management information databases. We also met with medical 
officers who performed medical assessments on behalf of the Department. We conducted audit work at the 
following missions: Beijing, Buffalo, Damascus, London, Moscow, and Paris. We also met with officials 
from other organizations during our visits to missions overseas. 


A survey was sent to 424 visa officers at missions abroad, and we received an overall response rate 
of 81.8 percent. Canada-based officers comprised 272 members of the survey population and provided a 
response rate of 79.0 percent, while locally engaged visa officers comprised 152 members and provided a 


response rate of 86.8 percent. 


At the CBSA, we interviewed officials at headquarters and examined documents, files, and databases. 
Furthermore, we examined statistically representative samples of temporary (109 cases) and permanent 


(50 cases) residence cases referred to the Agency for security screening that were finalized in 2010. These 
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samples are sufficient to conclude on their respective populations with a confidence level of 90 percent 


and a margin of error of +10 percent. 


Audit work was also conducted at RCMP and CSIS offices in Ottawa, mainly through interviews and 


review of key documents. 


Criteria 


Listed below are the criteria that were used to conduct this audit and their sources. 


To determine whether Citizenship and Immigration Canada and the Canada Border Services Agency have clarified the expectations and their respective 
roles and responsibilities under the Immigration and Refugee Protection Act to jointly manage the risks associated with the determination of admissibility, 
we used the following criteria: e ee : : 


Criteria 


Citizenship and Immigration Canada (CIC) and the Canada 
Border Services Agency (CBSA) have clarified the expectations 
and their roles and responsibilities for the inadmissibility 
provisions of the /mmigration and Refugee Protection Act. 


Sources 


¢ Immigration and Refugee Protection Act 


Immigration and Refugee Protection Regulations 


Memorandum of understanding between the CBSA 
and CIC, 2006 


Core Management Controls: A Guide for Internal Auditors, 
Office of the Comptroller General 


CIC and the CBSA have developed and implemented a joint 
integrated risk management strategy that identifies risks and 
mitigation strategies related to delivering the inadmissibility 
provisions of the Immigration and Refugee Protection Act. 


e Memorandum of understanding between 
the CBSA and CIC, 2006 


Framework for the Management of Risk, 
Treasury Board, 2010 


Integrated Risk Management Framework, 
Treasury Board, 2001 


Citizenship and Immigration Canada Integrated 
Risk Management Framework, 2002 


Corporate Risk Profile, Citizenship 
and Immigration Canada, 2010-2013 


Enterprise Risk Management Policy, 
Canada Border Services Agency, 2010 


To determine whether Citizenship and Immigration Canada and the Canada Border Services Agency provide visa officers with the necessary training, tools, 
and support to analyze the information required to make a well-informed admissibility determination before issuing visas, we used the following criteria: 


Criteria 


CIC and the CBSA have identified, developed and provided 
training, tools, and support to visa officers for the determination 
of admissibility. 


Sources 


¢ Memorandum of understanding between the CBSA and CIC, 
Information Sharing Annex, 2006 


° Core Management Controls: Guide for Internal Auditors, 
Office of the Comptroller General 


* Policy Framework for Information and Technology, 
Treasury Board, 2007 


28 Chapter 2 


Report of the Auditor General of Canada—Fall 2011 


ISSUING VISAS 


CIC’s Health Branch has systems and practices in place to 
provide visa officers with reliable and timely recommendations and Immigration Canada 
on medica! admissibility. . 


OP 15 Medical Procedures, Citizenship 


Designated Medical Practitioner Handbook, Citizenship 
and Immigration Canada 


CIC and the CBSA define and obtain, in a timely manner, useful ¢« Memorandum of understanding between CIC and CSIS, 2002 
information available on applicants from the RCMP and the 
Canadian Security Intelligence Service (CSIS). 


Memorandum of understanding between CIC and 
the RCMP. 2002 


Policy on Information Management, Treasury Board, 2007 


The CBSA has systems and practices in place to provide ¢« Memorandum of understanding between the CBSA 
intelligence and advice that support the decision-making needs and CIC, 2006 
of visa officers. 


A Vision for the Intelligence Program at the CBSA, 
October 2006, CBSA 


Management reviewed and accepted the suitability of the criteria used in the audit. 


Period covered by the audit 


With the exception of the audit work related to the memorandum of understanding between the CBSA 
and CIC, which came into effect in 2006, the period audited for this chapter is January 2010 to April 2011. 
Audit work for this chapter was substantially completed on 29 April 2011. 


Audit team 


Assistant Auditor General: Wendy Loschiuk 
Principal: Suzanne Therrien 
Director: Raymond Kunze 


Eve-Lyne Bouthillette 
Yan Lehoux 

Tammi Martel 
Catherine Martin 
Lisa Seguin 

Stacey Wowchuk 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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Appendix List of recommendations 


The following is a list of recommendations found in Chapter 2. The number in front of the 
recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses 


indicate the paragraphs where the topic is discussed. 


Determining admissibility 


2.36 The Canada Border Services 
Agency and Citizenship and 
Immigration Canada should ensure that 
operational manuals, risk indicators, 
and relevant country-specific 
information are complete, up to date, 
and made available to visa officers in a 
timely manner to help them identify 
foreign nationals who may be 


inadmissible. (2.23—2.35) 


The entities’ response. Agreed. A review of current risk 
indicators has begun in consultation with security partners. Also 
in consultation with security partners, a strategy will be 
developed to ensure that necessary country-specific information 
is available in a timely manner to support admissibility decision 
making. In addition, a review and update of policy manuals will 
be undertaken to ensure that accurate operational guidance is in 
place for all staff and management involved in the admissibility 
screening process. These measures will be completed by June 


2012. 


Security screening 


2.47 The Canada Border Services 
Agency should ensure that all 
information that can be obtained from 
security partners and is relevant to 
security screening is used to provide 


advice to Citizenship and Immigration 
Canada. (2.42-—2.46) 


The Agency’s response. Agreed. A comprehensive approach 
will be developed to ensure that all relevant information is 
available and shared with Citizenship and Immigration Canada 
(CIC). An assessment of information needs will take place in 
consultation with security partners, identifying any gaps and 
resulting in an information management strategy to ensure that 
necessary information can be accessed ‘to support admissibility 
decision making. In addition, existing information-sharing 
arrangements between CIC, the Canada Border Services 
Agency, the RCMP and the Canadian Security Intelligence 
Service will be reviewed to ensure that they are complete and 
serve the needs of the security screening program. These 
measures will be completed by March 2013. 
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2.63 The Canada Border Services 
Agency should 


* implement a quality assurance 
process to ensure the consistency and 
quality of information and advice 
provided by analysts to Citizenship 
and Immigration Canada (CIC); 


ensure that analysts have adequate 
training, support, and tools to perform 
their duties; 


establish service standards for the 
timeliness of security screening based 
on the needs of CIC and its own 
capacity and that of its security 
partners; and 


measure the effectiveness of its 
security screening to ensure that it 
meets the needs of CIC in a timely 
manner. (2.48-2.62) 


Medical screening 


2.17 In order to meet the objectives of 
the Immigration and Refugee Protection 
Act to protect the health and safety of 
Canadians, Citizenship and 
Immigration Canada should 


* develop a strategy based on risks to 
better identify applicants who present 
a danger to public health or a danger 
to public safety, and 


* examine the methodology and 
process for assessing excessive 
demand on health and social services 
and take corrective measures as 


necessary. (2.64—2.76) 
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The Agency’s response. Agreed. The Canada Border Services 
Agency has restructured its security screening program to 
provide more focus on quality assurance and performance 
management. A review of training requirements for screening 
officers is under way and a high-level plan has been adopted to 
provide screening officers with a competency-based training 
program and supporting tools, to be implemented incrementally. 
In consultation with Citizenship and Immigration Canada and 
security partners, the CBSA will review service standards and 
make necessary program adjustments to ensure that the needs of 
all partners are met. These measures will be completed by 


December 2012. 


The Department’s response. Agreed. Citizenship and 
Immigration Canada (CIC) has initiated and continues to 
develop a risk assessment and management strategy to better 
address danger to public health and danger to public safety in 
the immigration context. Among the activities conducted are 
the completion of a discussion paper on defining danger to 
public health, and the completion of an HIV policy review. 
CIC will implement a strategy that will result in consistent 
admissibility criteria, standardized processes, and improved 
monitoring of its programs. These measures will be completed by 
September 2013. 


CIC has started examining the excessive demand processes and 

will pursue its collaboration with the provinces and territories to 
review factors generating limitations and inconsistencies in the 

evaluation of excessive demands and will address the identified 

deficiencies. This measure will be completed by June 2013. 
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~ Recommendation Respons: 
2.81 Citizenship and Immigration The Department’s response. Agreed. Citizenship and 
Canada should implement a standard Immigration Canada (CIC) has already completed the 
quality assurance process to protect the | development of a Quality Assurance Framework for its 
integrity of the medical examination immigration medical examination and assessment programs. 
system and to ensure consistency and This framework will be integrated into the departmental 
quality in the assessment of medical Program Integrity Framework, providing the tool required to 
admissibility. (2.78-2.80) monitor and evaluate the quality of its immigration health 


program, worldwide, including the designated medical 
practitioners, other examiners, and CIC employees involved in 
the health assessment process. CIC is also in the process of 
implementing eMedical (an electronic medical system), enabling 
standardization and centralization of the medical examination 
process. These measures will be completed by March 2013. 


Governance, risk management, and performance measurement 


2.94 Citizenship and Immigration The entities’ response. Agreed. Citizenship and Immigration 
Canada and the Canada Border Canada (CIC) and the Canada Border Services Agency (CBSA) 
Services Agency should fully implement — will implement and monitor their joint risk management and 
their joint risk management and performance strategies for the shared delivery of the immigration 
performance measurement strategies and refugee program, including visa issuance. In this context, 


and monitor the results. (2.82—2.93) CIC and the CBSA will complete a review of Canada’s 
admissibility policy, in collaboration with security partners, in 
order to assess whether it is continuing to meet the needs of 
Canada and Canadians; to identify any gaps; and to recommend 
amendments to relevant legislation, regulations, or 
administrative guidelines. 


CIC and the CBSA will complete a joint priority-setting exercise 
based on key risks with respect to the intelligence program 
(including visa issuance) and resulting enforcement priorities. 
CIC and the CBSA will also complete work on key performance 
indicators with respect to visa issuance, which will be used to 
monitor the effectiveness of shared programs and to track the 
implementation of joint priorities. 


These measures will be completed by December 2012. 
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Chapter 


Payments to Producers— 
Agriculture and Agri-Food Canada 


Performance audit reports 


This report presents the results of a performance audit conducted by the Office of 
the Auditor General of Canada under the authority of the Auditor General Act. 


A performance audit is an independent, objective, and systematic assessment 

of how well government is managing its activities, responsibilities, and resources. 
Audit topics are selected based on their significance. While the Office may 
comment on policy implementation in a performance audit, it does not comment 
on the merits of a policy. 


Performance audits are planned, performed, and reported in accordance with 
professional auditing standards and Office policies. They are conducted by 
qualified auditors who 


¢ establish audit objectives and criteria for the assessment of performance; 
¢ gather the evidence necessary to assess performance against the criteria; 
* report both positive and negative findings; 

* conclude against the established audit objectives; and 


* make recommendations for improvement when there are significant 
differences between criteria and assessed performance. 


Performance audits contribute to a public service that is ethical and effective 
and a government that is accountable to Parliament and Canadians. 


Table of Contents 


Main Points 


Introduction 


Farm income support programs 
Focus of the audit 


Observations and Recommendations 


Improving program design 


Progress has been made on some design issues, but long-standing problems remain 


Improving program administration 

The Department has made progress on previous audit findings 

Monitoring of key planned actions needs improvement 

Further work is needed to improve the timeliness of payments to producers 
Monitoring federal/provincial delivery 


Transfer of AgriStability administration was sound, but monitoring responsibilities were not clear 
Audit and financial reporting requirements were substantially met 

Overall program performance is not publicly reported 

The Department is inconsistent in reporting on its performance to Parliament 


Managing the Tobacco Transition Program 


Key risks in program development were not considered y, 
The funding agreement between the Department and the Tobacco Marketing Board was not clear 


Follow-up initiatives were undertaken 


Conclusion 
About the Audit 


Appendix 


List of recommendations 
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What we examined 


Payments to Producers— 
Agriculture and Agri-Food Canada 


Main Points 


Agriculture and Agri-Food Canada (AAFC) has a mandate to provide 
information, research and technology, and policies and programs to 
achieve an environmentally sustainable, innovative, and competitive 
agriculture sector. The Department’s work includes supporting 
productivity and trade, stabilizing farm incomes, and conducting 
research. The federal government and the provinces and territories 
share responsibility for stabilizing farm incomes. Since 2008, they have 
done so through a joint federal/provincial/territorial agreement called 
the Growing Forward Framework Agreement. 


The Department’s programs have played an important role in 
supporting producers’ incomes when market income has dropped. 
Our audit looked at two programs under Growing Forward that, like 
the Canadian Agricultural Income Stabilization (CAIS) program they 
replaced, are aimed at protecting agricultural producers from drops in 
income. Agrilnvest is built around savings accounts, with producers’ 
deposits matched by government contributions; AgriStability is a far 
more complex program designed to protect against larger drops in 
income. Costs of the two programs total $1 billion annually, funded 
60 percent by the federal government and 40 percent by provinces. 
About 88 percent of payments under the AgriStability program are 
now delivered by the provinces. 


We also looked at the $284 million Tobacco Transition Program, aimed 
at replacing the quota system, helping tobacco producers move out of 


the tobacco industry, and improving the viability of those who remain. 


We looked at how the programs were developed, implemented, and 
administered. In particular, we looked at the quality of the risk 
assessment process and the process for continuous improvement. We also 
looked at the design and monitoring of funding and program delivery 
agreements between the Department and the provinces and territories or 
delivery organizations. We did not audit the provincial agencies or 


third-party organizations that participated in the delivery of programs. 


Audit work for this chapter was substantially completed 
on 2 May 2011. 
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Why it’s important 


What we found 


The Canadian agriculture and agri-food industry is vital to Canada’s 
economic success and its food supply. It encompasses several 
industries, including primary agriculture, input suppliers, food and 
beverage processing and distribution, and wholesale and retail food 
industries. According to the Department, this industry accounted for 
8.2 percent of total gross domestic product in 2009. 


The agricultural sector faces several challenges, including increasing 
international competition, rapid technological improvements, 
increased importance of environmental and health concerns, increasing 
input costs, rapidly evolving consumer preferences, changes in foreign 
exchange, and more volatility due to weather changes and disease. 


- To improve the design of its producer income support programs, 
Agriculture and Agri-Food Canada carried out industry 
consultation, a strategic review, producer surveys, and focus groups. 
Working with provinces and territories, the Department has made 
progress on some design issues, but long-standing concerns with 
AgriStability remain—clarity of program objectives, timely access to 
program funding, and program complexity that affects producers’ 
ability to reasonably predict payment amounts. 


The Department has made progress in addressing program 
administration issues raised in our 2007 audit—for example, 
payment accuracy and management of underpayments to producers. 
However, it has not systematically followed up on causes of delays 
in payments to producers in order to accelerate payments. Despite 
improvement, the Department has not yet met its AgriStability 
processing time targets, and producers can wait up to two years after 
an income loss to receive a payment. In addition, the Department 
has not collected information on processing times for Agrilnvest. 


Although federal/provincial/territorial accountabilities for 
performance reporting were not specified in the Growing Forward 
Framework Agreement, a performance measurement framework has 
since been agreed to by all governments. When fully implemented, 
overall national performance against service standards will be 
reported to Parliament and the public. With regard to assessing 
provincial capacity for AgriStability administration and transferring 
it to the provinces of British Columbia and Saskatchewan in 2010, 
the Department followed a sound process. 


The Department had to develop the Tobacco Transition Program 
within a short time frame and did not first conduct a thorough risk 
analysis. The agreement implementing the program did not provide 
clear terms and conditions to ensure that recipients would not enter 
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into business arrangements that would undermine the intent of the 
program. In addition, a number of times the Department changed its 
interpretation of what was and was not allowed under the Agreement, 
resulting in confusion for producers. As a result, the Department was 
successful in controlling some, but not all, business arrangements that 
it believed would undermine the intent of the program. 


The Department has responded. The Department agrees with all of 
our recommendations. Its responses follow each recommendation 
throughout the chapter. 
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Agricultural producers—Those who produce 


* livestock, poultry, related products (for 
example, eggs, dairy, and meat), and ranch 
fur products (for example, mink, fox); 


crop and plant products (for example, wheat, 
barley, oats, corn, fruit, nuts, vegetables, and 
tobacco); and 


nursery products (for example, trees, shrubs, 
annual and perennial plants). 
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Introduction 


Farm income support programs 


3.1 Agriculture and Agri-Food Canada has a mandate to provide 
information, research and technology, and policies and programs to 
achieve an environmentally sustainable, innovative, and competitive 
agriculture sector. The Department’s work includes supporting 
productivity and trade, stabilizing farm incomes, and conducting 
research. The Department has administered various farm income 
support programs since the late 1930s. The Department’s programs 
have played an important role in supporting agricultural producers’ 
incomes when market income has dropped. 


3.2 In 2008, the Department entered into a joint federal/provincial/ 
territorial agreement called the Growing Forward Framework 
Agreement. This five-year agreement expires on 31 March 2013, anda 
successor framework, called Growing Forward 2, is to be implemented 
on | April 2013. New framework agreements have to be negotiated 
and approved by federal/provincial/territorial governments. 


3.3. The Growing Forward Framework Agreement includes 

two programs aimed at protecting agricultural producers from drops in 
income. Agrilnvest is built around savings accounts, with producers’ 
deposits matched by government contributions. For example, if a 
producer deposits $1,000, the government will match this with $1,000. 
There is a maximum that each producer can deposit. AgriStability is a 
far more complex program designed to protect against larger drops in 
income. AgriStability payments are based on specific farm information, 
such as the number and type of crops and livestock, and farm sales and 
expenses over a number of years. 


3.4 Costs of the two programs total $1 billion annually and are funded 
60 percent by the federal government and 40 percent by provinces. 

For the 2009-10 fiscal year, the Department spent about $15 million 
administering AgriInvest and $40 million administering AgriStability. The 
provinces spent $44 million to administer AgriStability in the same year. 


3.5  Agrilnvest and AgriStability have significant similarities to 
two predecessor programs, the Net Income Stabilization Account 
(NISA) and the Canadian Agricultural Income Stabilization (CAIS) 
program. Exhibit 3.1 provides a summary of farm income support 
programs from 1991 to present. In 2008, AgriStability, together 
with Agrilnvest, replaced the coverage previously provided under 


the CAIS program. 
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3.6 Like the CAIS program, four provinces (Alberta, Ontario, 
Quebec, and Prince Edward Island) delivered AgriStability from the 
start. Saskatchewan and British Columbia assumed program delivery in 
January 2010. The six provinces now deliver about 88 percent of 
AgriStability payment dollars, and the Department delivers the 


remaining 12 percent of payment dollars to the other provinces. Prior 
to 2010, the Department delivered about 34 percent of AgriStability 
payments to producers. 


Exhibit 3.1 Summary of farm income support programs 


Annual EG 
Program years! program contribution | Number of participants Criteria | Delivery organization 
Net Income $347 million Over 144,466 Savings account matched by government Agriculture and 
Stabilization funds—proof of loss required to access Agri-Food Canada 
Account (NISA) | (1998-2002 eee funds (AAFC) 
average) average) 
1991-2002 
Canadian $1.3 billion 137,036 Covers income losses when farm income Provincial— 
Agricultural from current year is less than average farm | Alberta, Ontario, 
Income Soniye ee income from previous years Quebec, Prince 
Stabilization hile Ge eee : Edward Island 
(CAIS) Intended to help producers protect their 
operations from declines in income AAFC—all other 
2003-2006 provinces and 
territories 
Agrilnvest $279 million 140,894 Covers smaller income declines (15 percent | AAFC, Quebec 
(similar to NISA) ae (2007-2009 or less), previously covered under CAIS 
average) Savings account matched by government 
2007-2012 : 
+ funds—no proof of loss required to access 
$563 million Ms 
Kickstart (one- Limited to 1.5 percent of allowable net 
time payment in sales? 
2008) : 
Maximum payment is $22,500 per 
producer. 
AgriStability $731 million 109,870 Covers income declines greater than Provincial— 
(similar to CAIS) | (2007-2009 (2007-2009 ES Rete Dh providiehy covered-ence) GA)» aoe mei 
9007-2012 average) average) Covers income losses when farm income Echivard ilane 
from pe year is less than average farm (British Columbia 
income from previous years and Sadkatehewan 
Farm income calculations are based on as of January 
individual producer data and income tax 2010) 
Infonet cia AAFC—all other 
provinces and 
territories 


1 Based on program year. The processing of producer applications and payments occurs after the program or growing year is complete. 


Allowable net sales means eligible agricultural commodity sales net of allowable expenses. Eligible agricultural commodities are livestock and/or crops specified 
by the Department. 


Source: Agriculture and Agri-Food Canada 
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Stakeholders—tndustry groups representing 
agricultural sectors, such as dairy farmers and 
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3.7. The Department also implements one-time programs to address 
specific industry problems. One such program is the Tobacco 
Transition Program, which the federal government announced 

in 2008. Efforts have been undertaken previously, through two federal 
programs in 1987 and 2005 as well as complementary provincial 
programs, to help farmers exit from tobacco farming and improve the 
viability of remaining tobacco farmers. 


Focus of the audit 


3.8 Our audit included three programs involving payments to 
producers—Agrilnvest, AgriStability, and the Tobacco Transition 
Program. We examined whether the Department followed a risk-based 
approach in developing and implementing its payments to producers 
programs—in particular, whether it 


* followed a risk-based, lessons-learned approach in developing and 
improving the programs; and 


* established clear accountability and monitoring practices in 
implementing the programs. 


3.9 The audit focused on the process followed and action taken to 
develop, implement, and continuously improve the programs, 
including use of lessons learned, identification of risks and 
management of those risks, and use of program evaluation and 
strategic review findings. This included reviewing the Department’s 
progress in carrying out its commitments in response to the 
recommendations from our audit of the CAIS program in 2007. We 
also looked at the Department’s use of input from producers, partners, 
and stakeholders to improve program design and administration. As 
part of our audit, we looked at the process followed in transferring 
administration of AgriStability to the provinces of British Columbia 
and Saskatchewan. 


3.10 More details on the audit objectives, scope, approach, and 
criteria are in About the Audit at the end of this chapter. 
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Improving program design 


Observations and Recommendations 


3.11 Stabilizing farm incomes is a shared responsibility between the 
federal government, provinces, and territories. A key policy outcome 
of the Growing Forward Framework Agreement is that farm income 
support programs are to be timely, responsive, and predictable. 
Changes to program design need to be negotiated with the provinces 
and territories, and requirements for administering the programs are 
included in the Agreement and its Program Guidelines. 


3.12 The Treasury Board’s Integrated Risk Management Framework 
requires that plans, processes, and products be developed through 
ongoing consultation with stakeholders who may be affected by an 
organization’s decisions and actions. In our 2008 chapter entitled 
Managing Environmental Programming, we recommended that 
Agriculture and Agri-Food Canada prepare a schedule of review and 
consultation through the life of the Growing Forward Framework 
Agreement to ensure a smooth and timely transition to future policy 
frameworks. The Department agreed and indicated that it would also 
assess the strengths and weaknesses of past approaches. 


3.13 We examined a number of initiatives carried out by the 
Department to improve the design of two producer income support 
programs, AgriStability and Agrilnvest. These initiatives included 
industry consultation, a strategic review, producer surveys, and focus 
groups. Some of these initiatives were undertaken with the 
transformation of the Canadian Agricultural Income Stabilization 
(CAIS) program to AgriStability and Agrilnvest. Other more recent 
initiatives have been carried out in preparing for the next income 
support framework—Growing Forward 2, to be implemented in 2013. 


3.14 During the life cycle of the CAIS program (2004-2007 program 
years), the Department carried out the following initiatives in the 
redesign to AgriStability and Agrilnvest: 


* The CAIS Task Team was formed to provide program 
options (2006). 


¢ Industry consultations were held (2006-07). 


In addition, there was input from two joint government/industry 
standing committees—the National CAIS Committee and the 
National Safety Nets Advisory Committee. 
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3.15 To provide input to the development of Growing Forward 2, 
the Department carried out the following additional initiatives: 


* A survey was conducted to gather producer feedback on 


service (2009). 


¢ A survey was conducted on farm risks and how the farm income 
support programs are helping (2010). 


* Focus groups on service quality were held (2009-10). 


* A strategic review was conducted to address continuing pressures 
for program changes (2008-10). 


* As well, there has been ongoing input from a new joint 
government/industry committee—the National Program 
Advisory Committee. 


Progress has been made on some design issues, but long-standing problems 
remain 


3.16 Improved design. The CAIS Task Team proposed three program 
design changes, which were approved by federal/provincial/territorial 
ministers. These resulted in a more accurate assessment of producers’ 
inventories (numbers of livestock and/or harvested crops on hand), 
improved payments to producers facing back-to-back losses, and an 
early partial AgriStability payment option for producers experiencing a 


oS 


regional or sector-wide disaster. 


3.17 During the industry consultations held in the 2006-07 fiscal 
year, producers indicated that some portion of a producer’s income 
loss should be replaced with a program similar to the earlier NISA- 
type program (producers’ savings account deposits matched by 
government). The Agrilnvest program was therefore implemented to 
replace the support previously provided by the CAIS program for the 
first 15 percent of income loss with a savings account program. 
Producer deposits are matched by contributions from the federal and 
provincial governments. Agrilnvest amounts received by producers 
are more predictable and simpler to administer than the previous 
CAIS payments. 


3.18 Delayed implementation of program. The Department planned 
to have an agreement with financial institutions in the fall of 2008, 

so that Agrilnvest accounts would be in place by 2009. However, 
financial institutions were reluctant to take part in the program as 
presented, in part because producers could withdraw the funds at any 
time (which could limit the recovery of the costs of setting up 


Agrilnvest bank accounts). 
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3.19 Financial institutions were also concerned that the Department 
might waive the producer deposit requirement, as it had done for 

the 2003 to 2005 CAIS program years. This would again limit cost 
recovery from account balances. In addition, the government proposed 
a new Information Technology (IT) transfer system that was not 
compatible with their systems, instead of using existing IT systems that 
were compatible. Given the economic climate at the time, the 
financial institutions were reluctant to incur the infrastructure costs to 
set up the accounts, only to have the deposit requirement waived and 
the program potentially end a few years later. 


3.20. In addition, financial institutions found the Department’s 
timeline for preparing to deliver the Agrilnvest program unrealistic. 
As a result, the Department had to administer the AgriInvest accounts 
until financial institutions were ready to take them over. As of 

May 2011, more than one third of producer accounts, representing 

12 percent of account balances, were still being administered by the 
Department and not by the financial institutions. For the first 

three program years of a five-year program, producers had different 
processes for participating in Agrilnvest. 


3.21 Despite the delay in implementing the program through financial 
institutions, producers benefited because they were not required to 
make a matching deposit. In 2008, producers received about 

$563 million in AgriInvest Kickstart funds (Exhibit 3.1), with no 
matching deposit required. As well, for the 2007 program year, the 
government's share was provided, but the Department waived the 
deposit requirement for producers. For the 2008 and 2009 program 
years, a deposit by producers was required to obtain the government’s 
matching funds. . 


3.22 Long-standing problems. The Department conducted 

two surveys and a set of focus groups in 2009 and 2010. Survey 
participants indicated that farm income support programs helped them 
to minimize financial risk. However, the survey reports indicated that 
for AgriStability, 


* users were less satisfied than users of other farm income support 
programs; 


* timeliness of payments to producers was a concern; 


* producers found that fulfilling program requirements was 
burdensome; 
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* barriers to producers applying to the programs were related to the 
complexity of the program, lack of payment predictability, and 
cost of preparing program applications; and 


* program information was easy to find but difficult to understand. 


3.23. Producers have previously raised similar concerns about the 
CAIS program. For complex programs like CAIS and AgriStability, 
where the payment is based on specific farm information over a 
number of years, there is an inevitable trade-off between program 
specificity and timeliness of payment. The more tailored the payment 
is to the farm operation, the more complex the calculation becomes 
and the longer it takes to process the application. The Department, 
with the provinces, has to find the balance between these trade-offs. 


3.24 The Department informed us that it has received some input on 
these trade-offs from producer representatives. Obtaining this 
feedback is important for the Department to be aware of the priority of 
producers’ concerns and to help manage producers’ expectations for 
program modification. 


3.25 The Department has presented various options to address long- 
standing concerns of producers to the federal/provincial/territorial 
working groups that oversee the income support programs. Despite the 
initiatives to improve program design, efforts to address the long- 
standing issues of program complexity, predictability, and timeliness of 
payments to producers have had only limited success. It may be 
necessary to further engage the agricultural sector industry 
representatives and producers to ensure that trade-offs between 
targeting and timeliness are understood. 


3.26 Another long-standing problem is understanding program 
objectives and responsibilities for managing farm income risks. 

From 2008 to 2010, the Department conducted a strategic review to 
address pressures from producers and industry for program 
improvements. The review found a lack of clarity in the roles and 
responsibilities for producers, industry, and government relative to 
farm income risk management. It also found that objectives for income 
support programs needed to be clarified. The review concluded that 
these factors may have contributed to the current discontent and 


differing expectations of producers. 


3.27 In summary. We found that the Department is aware of and 
has identified concerns with its income support programs. Federal 
provincial/territorial governments have made some improvements to 


program design—for example, by introducing Agrilnvest. However, 
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long-standing concerns remain, including unclear program objectives, 
lack of timely access to program funding, and program complexity that 
affects producers’ ability to reasonably predict payment amounts. 
Department officials have indicated that they are aware of these 
concerns and are discussing their implications during their 
consultation with industry and provincial/territorial partners for 
Growing Forward 2 programs. 


3.28 Recommendation. Agriculture and Agri-Food Canada should 
work with provinces and territories to help increase producers’ and 
stakeholders’ understanding of the objectives and trade-offs of income 
support programs and to clarify government’s and producers’ 
responsibilities for managing farm risks. 


The Department’s response. Agreed. The current suite of programs 
was designed with industry to respond to the needs of producers. In 
conjunction with provincial and territorial governments, the 
Department continues to consult extensively with industry in the 
development of programming, most recently through two rounds of 
industry engagement on the next generation of the agricultural policy 
framework (Growing Forward 2). Through future rounds of broad- 
based engagement, as well as more targeted sessions with producers, 
their associations, and formal mechanisms such as through the 
National Programs Advisory Committee, the Department will 
continue to gain a common understanding with producers of the 
respective roles in managing risk, of the objectives of programming, 
and of the inherent trade-offs within programs. This recommendation 
will be fully implemented with Growing Forward 2 in April 2013. 


3.29. Recommendation. Agriculture and Agri-Food Canada should 
work with service delivery partners to understand their challenges and 
develop and monitor realistic implementation timelines for future 
income support programs. 


The Department’s response. Agreed. In delivering programs, the 
Department’s objective is to ensure that the intended benefits of 
programming are available to participants in a timely manner. Under 
the Growing Forward 2 policy framework, the Department will actively 
engage service delivery partners in the process of developing and 
implementing income support programs. Officials will work with these 
partners to ensure that challenges are identified and documented, and 
that workable and timely solutions are developed to secure appropriate 
service delivery for producers. As it did with the delivery of Agrilnvest, 
the Department will also ensure that contingency plans are in place so 
that producers have access to program benefits despite challenges that 


12 


| Chapter 3 


Report of the Auditor General of Canada—Fall 2011 


Improving program administration 


PAYMENTS TO PRODUCERS—AGRICULTURE AND AGRI-Foop CANADA 


may arise with the intended delivery arrangement. By relying on such 
contingency plans, the Department was able to ensure that producers 
had access to Agrilnvest funding, although the infrastructure with 

financial institutions was not in place for the 2007 and 2008 program 
years. This recommendation will be fully implemented with Growing 


Forward 2 by April 2013. 


3.30 In our 2007 audit, we raised concerns about Agriculture 

and Agri-Food Canada’s administration of the CAIS program. 

The Treasury Board’s Framework for Service Improvement 

requires departments to have systems in place aimed at continuous 
improvement. As well, the Treasury Board’s Risk Management Policy 
requires departments to analyze and assess the risks identified and to 
implement cost-effective prevention, reduction, or avoidance control 
measures. We examined the Department’s progress on previous audit 
findings, and its monitoring of evaluation recommendations, risk 
management strategies, and causes of delays to timely processing. 


The Department has made progress on previous audit findings 


3.31 We found that the Department has made progress on some key 
administrative issues raised in our 2007 CAIS audit. At that time, we 
noted that the Department’s method of calculating payments (also 
referred to as benefits in the Department) was not clear to producers. 
Our current audit found that the Department has made several 
improvements to the Calculation of Benefit Statement, such as 
including a summary and explanation of changes to submitted 
application forms. 


3.32. In 2007, we found that the Department’s focus on overpayments 
to producers was not proportional to the risk of producer 
underpayments. In our current audit, we found that the Department 
has made enhancements to its policies and procedures to better 
balance the identification and management of over- and 
underpayments, including improvements to the automated validation 
of applications and the audits of producers. 


3.33 For AgriStability, part of the Department’s quality control system 
is to review samples of payments after they have been issued to assess 

whether payments have been accurately calculated. In our 2007 audit, 
we found that the accuracy rate was close to the performance target of 


97 percent; our current audit found that the payment accuracy rate for 
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3.34. The Department also reviews samples of AgriStability payments 
before they are issued to determine the error rate and correct the 
payments before issuing them. We found that pre-payment errors have 
decreased, from 12.3 percent in 2005 to 3.7 percent in 2008. 


Monitoring of key planned actions needs improvement 


3.35 Program evaluation. An evaluation of the CAIS program was 
reported in 2007. The evaluation report noted that its scope was limited 
by the lack of recent performance information, by the lack of survey 
data to provide insights into producer feedback, and by other things. 


3.36 Asa result, the evaluation recommended a number of 
administrative improvements, such as the development of more timely 
performance information and data showing the link between income 
support programs and producers’ ability to manage risks and be 
profitable and viable in the long term. 


3.37 The Department established an action plan subsequent to 

the evaluation. All actions were to be completed by 2008; however, 
this was not achieved. We found a lack of systematic tracking of 
progress against the planned actions and little progress on some 
recommendations compared with others. For example, while the 
Department developed a sound performance measurement framework 
in August 2009 for the successor program AgriStability, it has made 
little progress in improving communication to producers on their farm 
risk management responsibilities. 


3.38 Risk mitigation. We found that program risk assessments were 
prepared for AgriStability and Agrilnvest. Identified risks included 
producer confusion about the transition from the CAIS program to 
AgriStability and Agrilnvest, and impacts on the Agrilnvest program 
related to potential delays in developing new computer systems. 


3.39 We found that there was no formal mechanism to track actions 
against mitigation strategies at the program level, and clear dates for 
carrying out mitigation strategies were not specified. Some of the 
identified risks materialized, such as continued producer confusion 
about the payment calculation for AgriStability. Delineation of risk 
management activities and timelines as well as periodic monitoring of 
progress are essential to mitigating risks on a timely basis. 


3.40 Recommendation. Agriculture and Agri-Food Canada should 
develop and implement formal tracking and reporting systems to 
ensure that key planned actions for AgriStability and Agrilnvest are 
carried out as expected. 
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The Department’s response. Agreed. As a matter of practice, we 
work with provinces and territories to monitor, assess, and evaluate 
programs with a view to continuous improvement. The development 
of farm income support programs under the current Growing Forward 
Framework Agreement was informed by audits, evaluation, reviews, 
and assessments. As this type of work is undertaken for the next 
framework agreement, officials will implement a formal system to track 
documents to demonstrate the actions taken to address the 
recommendations, proposed actions, and risk-mitigating strategies that 
are carried out. This recommendation will be fully implemented with 


Growing Forward 2 by April 2013. 


Further work is needed to improve the timeliness of payments to producers 


3.41 AgriStability. The Department established a service standard for 
processing 75 percent of applications within 75 calendar days of 
receiving a complete application. Calculation of processing time does 
not start until the Department has received a completed program 
application, the required fees, and the relevant information from the 
Canada Revenue Agency. The Department has yet to meet its time 
standard for AgriStability payments, although there has been 
improvement over time. According to how the Department reports its 
performance against the service standard, its reported performance 
improved significantly from 38 to 66 percent, from the 2005 to 

the 2008 program year. 


3.42 However, we noted that the Department publicly reports only on 
applications processed in determining its success in meeting the service 
standard. If it included the backlog of applications that have not yet 
been processed, its monthly success rate would drop. For example, in 
January 2011, 23 percent of processed applications for the 2009 
program year met the 75-day standard. However, when unprocessed 
applications already beyond the 75-day standard were included, only 
11 percent met the 75-day service standard. 


3.43 We found that it could take up to two years for a producer to 
receive a payment, subsequent to an income loss. A recent analysis of 
payment timeliness conducted by the Department shows that 

55 percent of producers are paid within 9 months following the year 
of loss, and 85 percent are paid within 19 months of the year of loss. 
The Department has completed some analysis to identify the causes of 
payment delays. However, it has not systematically followed up on the 
causes of delays and identified remedies to accelerate payment 


timeliness. 
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3.44 To provide more timely support, the Department provides an 
interim AgriStability payment option. This provides earlier access to 
50 percent of the estimated payment. However, use by producers has 
been low at 5.2 percent for the 2009 program year and 1.6 percent 
for the previous year. Furthermore, we found that the majority of 
interim AgriStability payments were not being processed within the 
Department’s 30-day time standard. Despite this delay, the early 
AgriStability payment mechanism has been helpful in assisting some 
producers—for example, hog producers—in dealing with a sector-wide 
disaster (for example, the H1IN1 virus and trade barriers to Canadian 
exports). 


3.45 AgriInvest. Providing timely and predictable income support 
was a key reason for implementing Agrilnvest. The application process 
for Agrilnvest requires the producer to submit an application and, 
once it is approved, deposit funds in the account. These funds are then 
matched by the Department. Since the program was introduced 

in 2007, performance reports have not included information on 
payment timeliness or accuracy for Agrilnvest. Without this 
information, the Department does not know whether Agrilnvest 
payments to producers are timely. 


3.46 Recommendation. Agriculture and Agri-Food Canada should 
analyze processing of payments for AgriStability and Agrilnvest in a 
more systematic manner and follow up on remedies to improve the 
timeliness of payments to producers. 


The Department’s response. Agreed. The Department is aware of the 
concerns about the timeliness of program payments. Leading up to the 
implementation of the current suite of programs, the Department 
reviewed potential approaches to improving the timeliness of payments 
in a number of venues with industry, as well as with provincial and 
territorial governments. The Department will more systematically 
review the design of income support programming as part of Growing 
Forward 2 discussions and will analyze its payment processes to 
improve the timeliness of processing of payments. In that regard, the 
recent implementation of an updated application processing system 
will provide the Department with the capacity to produce more 
systematic reporting on application processing. The Department will 
use the reporting capacity to further improve the timeliness of 
producer payments. This recommendation will be fully implemented 


with Growing Forward 2 by April 2013. 


16 


Chapter 3 


Report of the Auditor General of Canada—Fall 2011 


Monitoring federal/ 
provincial delivery 


Business case—Within a Government of 
Canada context, a business case is a proposal 
by an organization seeking approval for an 
initiative or project. It provides the information 
necessary to make an informed decision about 
whether to proceed. It is also the basis against 
which outcomes will be compared and 
evaluated 
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3.47 The Growing Forward Framework Agreement allows either 
federal or provincial administrations to deliver the AgriStability 
program. In accordance with the Treasury Board’s Directive on 
Transfer Payments, departments are to ensure that delivery 
organizations or provinces are able to deliver federal programs and 
that agreements include written accountabilities that are monitored. 


3.48 A number of provinces (Alberta, Ontario, Quebec, and Prince 
Edward Island) administered the CAIS program and subsequently the 
AgriStability program when it came into effect. They delivered about 
60 percent of the AgriStability payments. 


3.49 In October 2009, the Government of Canada approved funding 
to transfer the administration of AgriStability to British Columbia and 
Saskatchewan. The provinces indicated that they could provide better 
and more efficient service due to their knowledge of and proximity to 
producers. By December 2009, agreements with the two provinces 
were signed and producers were formally advised of the transfer. As of 
January 2010, the two provinces assumed responsibility for processing 
all applications in 2009 and subsequent program years. As a result, 
about 88 percent of AgriStability payment dollars are now delivered by 
the six provinces, and Agriculture and Agri-Food Canada delivers the 
remaining 12 percent of payment dollars to the other provinces. 


3.50 We examined the transfer of the administration of the 
AgriStability program to the provinces of British Columbia and 
Saskatchewan, and whether financial and performance reporting 
requirements for both AgriStability and Agrilnvest were met. 


Transfer of AgriStability administration was sound, but monitoring responsibilities 
were not clear 


3.51 Transferring AgriStability administration. We examined 
whether the Department ensured that the provinces of British 
Columbia and Saskatchewan had the capacity to deliver the program. 
Provinces were required to formally express interest in the delivery of 
AgriStability and then to present a business case to support their 
request. British Columbia submitted its business case in February 2009 
and Saskatchewan in May 2009. 


3.52 We found that the Department and provinces consulted 
extensively on the business cases to ensure that they had an 
understanding of their responsibilities in delivering the AgriStability 
program. The provinces engaged third parties to review their business 


cases. The Department analyzed the provincial business cases and the 
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third-party reviews of them. It also ensured that the provinces had the 
capacity to deliver the programs—for example, it ensured that the 
provinces hired and trained sufficient numbers of staff to successfully 
deliver the AgriStability program. 


3.53 The provinces committed to improving customer service and 
reducing administrative costs over time. These commitments included 


¢ improved collection of customer service feedback, 
¢ reduced administrative costs, 
* improved communication with clients, and 


¢ audits of payments to producers conducted by provinces. 


3.54 Monitoring provincial commitments. The Department reviews 
administration and program costs to verify. its 60 percent share of 
overall costs. However, we found that roles and responsibilities were 
not clear for post-transfer monitoring of all the commitments made by 
the provinces. It was not clear whether the provinces or the federal 
government should be reporting on these commitments and whether 
reporting should be only internal or also to the public. 


3.55 Recommendation. Agriculture and Agri-Food Canada should 
add, to its future business case requirements, the need for a proposal 
from the provinces on whether the federal government or provincial 
governments will monitor and report on commitments in the 
business case. 


The Department’s response. Agreed. The Department will clarify 
how the monitoring and reporting of key provincial commitments (not 
already covered by existing mechanisms) should be addressed in any 
future business cases. 


Audit and financial reporting requirements were substantially met 


3.56 The Growing Forward Framework Agreement includes some 
monitoring provisions that enable each of the parties to confirm that 
the program is being administered according to the terms and 
conditions of the Agreement. These include preparing audited financial 
reconciliations of program payments and administrative expenses and 
carrying out compliance audits of provincial administrators. 


3.57 We found that federal and provincial administrators had 
completed the audited reconciliations of program payments and 
administrative costs for the year ended 31 March 2009 within or 
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close to the time limit stipulated in the Growing Forward Framework 
Agreement. 


3.58 The last time the Department carried out compliance audits of 
all program delivery administrators was for the 2005 CAIS program 
year. The Department told us that starting in the 2010-11 fiscal year, it 
planned to synchronize AgriStability compliance audits with audits of 
another federal farm program also administered by the provinces. Plans 
to audit one to three provincial administrations per year mean that, for 
some provincial administrators, there could be a six- to seven-year gap 
between compliance audits. Potential problems in program 
administration may not be known for many years, thereby preventing 
timely improvements to current and future programs. 


Overall program performance is not publicly reported 


3.59 The Department is delivering about 12 percent of program 
payments for AgriStability—a forecasted $100 million for the 

2010 program year, while the provinces are delivering the remaining 

88 percent—about $705 million. The Department delivers payments for 
Agrilnvest in all provinces except Quebec. It is delivering $254 million 
while Quebec is delivering about $35 million (about 12 percent). 


3.60 The Growing Forward Framework Agreement was signed by all 
provinces. According to the Agreement, each party is to be subject to 
public scrutiny, and for reporting purposes, its work must be managed 
in a manner that is open and transparent to the public. However, the 
Agreement does not specify the nature and frequency of performance 
information to be shared by the parties to the Agreement, or the type 
of information to be reported on publicly. 


3.61 Despite these requirements not being in the Growing Forward 
Framework Agreement, Agriculture and Agri-Food Canada has made 
progress in designing a performance measurement framework. The 
Department’s new performance measurement framework for farm 
income support programs has several performance indicators for 
Agrilnvest and AgriStability. These include some important outcome 
measures, such as the extent to which income support programs 
respond to producer income losses. The Department and the provinces 
have agreed to the performance measurement framework, and it has 
been shared with a joint government/industry committee. Agreement 
still needs to be obtained with the provinces on some service standards 


for public reporting. 
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3.62 The Department has collected and reported internally on 
performance information on agreed-upon service standards. However, 
there was an extensive delay in reporting overall results internally. 
Performance results for a program year cannot be reported until 
processing of applications is complete. Due to delays in application 
processing, the first internal performance report was not released until 
April 2010 for the 2007 program year. Performance results for the 2008 
program year were reported in January 2011. 


3.63 Given that the performance measurement framework has not yet 
been fully implemented, the Department does not provide information 
to the public about overall performance in administering AgriStability, 
including processing time, accuracy, and administrative cost efficiency. 
However, it does provide some overall information on producers 
participating in the program, such as program participation by size of 
farm and commodity type. Our recommendation is found at 
paragraph 3.66. 


The Department is inconsistent in reporting on its performance to Parliament 


3.64 We reviewed the Department’s departmental performance 
reports and associated supplementary tables for the last three fiscal 
years (2007-08 to 2009-10) related to the CAIS/AgriStability Program 
to assess whether program results have been consistently identified. 


3.65 We found that the reporting of targets and actual results was 
inconsistent, making year-to-year comparisons very difficult. Some 
targets and results were included in some years but not in others. Some 
of the inconsistency was due to the introduction of new programs 
under the Growing Forward Framework Agreement and changes in 
government-wide reporting requirements. The Department informed 
us that it will consistently report on its new performance measurement 
framework in the future. 


3.66 Recommendation. Agriculture and Agri-Food Canada should 
work with the provinces to finalize and fully implement the agreed- 
upon performance measurement framework and to improve the 
completeness and timeliness of reporting to Parliament and the public 
on income support programs. 


The Department’s response. Agreed. The Department is committed 
to measuring and reporting on program performance and has 
developed an income support programs measurement framework with 
the provinces and territories to do so. The Department will refine the 
framework and leverage it in its departmental performance reporting 
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in this chapter, the authorized amount of tobacco 
that can be grown. 
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so that the reports are consistent and provide Parliament and the 
public with a complete view of program performance. This 
recommendation will be fully implemented with Growing Forward 2 


by April 2013. 


3.67 Agriculture and Agri-Food Canada has frequently had to 
respond to an agricultural need by means of a one-time program. 
Particular and tailored strategies are therefore needed to deal with the 
unique challenges posed by this type of program. 


3.68 Our audit examined one such program—the Tobacco Transition 
Program. This program is not an ongoing farm income support 
program. It was created to address the severe decline in demand for 
flue-cured tobacco from southern Ontario, which is the primary 
region in the country for producing flue-cured tobacco. 


3.69 The Ontario Flue-Cured Tobacco Growers’ Marketing Board 
(Tobacco Marketing Board) was established in 1957 under provincial 
jurisdiction, as an entity authorized by Ontario law to regulate the sale 
of flue-cured tobacco leaf. The Board determined the price at which 
flue-cured tobacco would be sold to manufacturers, and it controlled 
the amount of flue-cured tobacco that could be grown through a quota 
system (at that time farmers were given a quota (pounds) for flue- 
cured tobacco production). 


3.70 With declining production, the control of producers’ entry/exit 
into the sector, and production and marketing of tobacco through a 
quota system, was no longer seen as the most appropriate way to 
manage production in the tobacco sector. Repealing the quota system 
and providing transition assistance would help to further rationalize 
the industry and allow tobacco producers to negotiate the price of their 
tobacco, similar to other agricultural commodities. 


3.71 In 2008, the federal government announced the Tobacco 
Transition Program (TTP). The program had three objectives: 


* Remove the supply-managed quota system and implement a 
licensing system. 


* Facilitate the transition of Ontario flue-cured tobacco producers 


to exit the tobacco industry. 


* Improve the viability of remaining and future tobacco producers 


in southern Ontario. 


3.72 The Tobacco Marketing Board delivered the TTP to producers 


on behalf of the Department. Under this program, tobacco producers 
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who agreed to leave the industry were paid $1.05 per pound of quota. 
The Tobacco Marketing Board paid out a total of $284 million by 
May 2009. The provincial government did not contribute funds to 
the TTP 


3.73. About 1,000 quota holders received payment under the TTP 
and more than half were not active tobacco producers at the time. 
In 2008, 2009, and 2010, there were 446, 118, and 251 active tobacco 


producers, respectively. 


3.74 The terms of the TTP required the Tobacco Marketing Board to 
remove the quota system and replace it with a licensing system. A 
licensing system does not limit the amount of tobacco that a producer 
can grow. Rather, to obtain a licence, a producer of flue-cured tobacco 
requires a contract of sale with a licensed buyer. 


3.75 According to the Treasury Board’s Integrated Risk Management 
Framework, departments are to apply risk management and lessons 
learned in program design and implementation. In addition, in 
accordance with the Treasury Board’s Directive on Transfer Payments, 
departments are to ensure that external organizations are able to 
deliver federal programs and that agreements include written 
accountabilities that are monitored. 


3.76 The Tobacco Transition Program was developed and delivered 
within a short time frame. The Department received authority to enter 
into negotiations with the Province of Ontario in July 2008. The 
government announced the program publicly on | August 2008. 
Payments to producers were to be made by | May 2009. It was a 
challenge to establish design and delivery teams, obtain the 
appropriate authorities, develop the Funding Agreement with the 
service provider, and deliver the program in such a short time frame. 


3.77 In addition, producers needed to be informed of the program 
details early to make important business decisions on their 
participation and on whether they should grow tobacco that year. 
Greenhouses need to be seeded in March, and tobacco seedlings are 
planted in the field in early May. Producers had to apply to the TTP 
by 23 March 2009. 


3.78 We examined the Department’s risk assessment for the TTR We 
also reviewed the Canada-Ontario Tobacco Board Funding Agreement 
(Funding Agreement) for the program. We reviewed correspondence 
between the Department and the Tobacco Marketing Board, program 
information to recipients and the public, and other departmental 
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documentation. Our recommendations for this program appear at 


paragraphs 3.82 and 3.91. 


Key risks in program development were not considered 


3.79 The Department did not conduct a thorough risk assessment of 
the Tobacco Transition Program. However, it carried out various 
actions to manage risks as they arose. It initially identified only one risk 
related to the TT P—that the quota system for flue-cured tobacco 
might not be removed and replaced by a licensing system within the 
required time frame. We did not see evidence that the importance of 
other risks was recognized prior to developing and signing the Funding 
Agreement in February 2009. Examples of other important risks are 
the following: 


* Inappropriate quota transfers. For the previous Tobacco 
Transition Assistance Program in 2005, tobacco farmers were not 
allowed to transfer their quota once the government announced 
the program. Transferring quota could allow someone who was 
never involved in tobacco farming to obtain the transition 
assistance payment, while the original quota holder continues to 
farm tobacco. The Department did not anticipate this risk or the 
extent to which it could occur. 


* Confusion about program terms. Another key risk that was not 
identified at the outset was confusion about program terms among 
recipients and the Tobacco Marketing Board. Throughout 
program implementation and afterwards, the Department 
responded to many complaints and queries from producers, 
lawyers, and others. The majority related to the business 
arrangements that TTP recipients could enter into for tobacco 
production and to the eligibility rules to obtain a licence to grow 
tobacco. The Department was still clarifying program terms for 
acceptable business arrangements in May 2009. 


3.80 As well, the Department did not develop a formal risk register to 
identify and monitor risks during delivery of the program for follow-up 
on actions. However, it reacted quickly to address problems, to the 


extent possible, when they arose. 


3.81 The lack of a thorough risk assessment and associated risk 


mitigation strategies had significant consequences: 


* Key program terms were not defined on a timely basis for 


producers and the Tobacco Marketing Board. 


* Inappropriate quota transfers took place and had to be reversed. 
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¢ The Department was successful in controlling some, but not all, 
business arrangements. 


¢ Unnecessary confusion about acceptable business arrangements 
persisted, such as the type of arrangements that were allowed 
between TTP recipients and tobacco producers, including close 
relatives. 


3.82 Recommendation. For future one-time payment programs, 
Agriculture and Agri-Food Canada should conduct a thorough risk 
assessment during program development and maintain a risk register to 
support corrections throughout program delivery. 


The Department’s response. Agreed. The Department recognizes the 
need to conduct thorough risk analysis and develop mitigation 
strategies commensurate with program risks when implementing 
one-time payment programs. Consistent with the Policy on Transfer 
Payments, the Department is adopting a risk-based management 
approach to program design and implementation. Additional guidance 
and tools are under development to guide the development of program 
risk registers and ongoing risk management practices. This 
recommendation will be fully implemented by December 2012. 


The funding agreement between the Department and the Tobacco Marketing Board 
was not clear 


3.83 The Canada-Ontario Tobacco Board Funding Agreement did 
not provide clear expectations to the Tobacco Marketing Board on its 
administration of the Funding Agreement. In particular, it did not 
prevent quota transfers, and it was also not explicit about the types of 
relationships allowed between TTP recipients and future tobacco 
producers so that the intent of the program would not be undermined. 
The Department of Finance and the Treasury Board of Canada 
Secretariat recommended the addition of wording to control 
transactions between related individuals. However, Agriculture and 
Agri-Food Canada used very broad wording when it drafted the 
Funding Agreement, which was subsequently approved by the 
government. 


3.84 Quota transfers. In March 2009, the Department became aware 
that a large number of quota transfers to non-quota holders were being 
exercised. It received complaints that these quota transfers were being 
done to bypass the TTP conditions. In response to these complaints as 
well as evidence that tobacco producers were transferring quota to 
non-farming family members, the Department directed the Tobacco 
Marketing Board to scrutinize all quota transfers and to deny a licence 
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to anyone whose actions could undermine the purpose and intent of 
the transition program. The number of quota transfers was reduced 
from 336 to 22, and quota holders incurred additional legal and 
accounting expenses in reversing the quota transfers. 


3.85 Business arrangements. On 9 April 2009, the Department 
attempted to prevent the Tobacco Marketing Board from allowing 
certain types of business arrangements, including the following: 


* TTP recipients working on their own farm as employees; 


* independent children of TTP recipients renting their parents’ 
land and infrastructure, and obtaining a licence to grow tobacco; 
and 


* TTP recipients loaning money to a licensee or co-signing at a 
bank for a loan for a licensee. 


Without clear wording in the Funding Agreement, the Department 
could not impose its interpretation of the wording on the Tobacco 
Marketing Board. Therefore, the Department had to reverse its 
position on these issues on 7 May 2009. This led to additional 
confusion among producers. 


The series of events listed in Exhibit 3.2 shows that the Department 
did not adequately prepare for the range and type of risks that might 
affect a program of this nature. 


Follow-up initiatives were undertaken 


3.86 External audit. After the TTP was implemented, the 
Department engaged external auditors to ensure that the TTP 
recipients, as well as the Tobacco Marketing Board, had complied with 
the obligations in the Funding Agreement. The external auditors 
ensured that the quota system was removed and verified the amount 
transferred from the Department to the Tobacco Marketing Board. In 
addition, they reviewed documents related to business arrangements 


with TTP recipients. 


3.87 The external audit found that the Tobacco Marketing Board “did 
not appear to have issued licences to TTP recipients, their spouses or 

dependent children.” In addition, the external auditors found that 

13 (8 percent) business arrangements in 2009 and 56 (15 percent) 

in 2010 did not appear to be at fair market value. As well, 11 producers 
did not respond to the auditors’ phone calls requesting further 


information. 
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Exhibit 3.2 Agriculture and Agri-Food Canada did not adequately prepare for the risks that might affect a program such as the Tobacco 


Transition Program 


Action 


The Department announced the Tobacco 
Transition Program (TTP). 


Date 


1 August 2008 


Action details 


The Department issued a fact sheet announcing the TTP 
and stating that producers who accepted transition 
assistance would not be able to re-enter tobacco 
production. 


Producers submitted a petition requesting 
information on the terms and conditions of the 
program. 


15 December 
2008 


As a result of the petition, the Tobacco Marketing Board 
arranged a meeting for its membership in February to 
help producers make informed choices. This was a joint 
meeting presented by the Department and the Tobacco 
Marketing Board. 


The Department issued a news release/fact 
sheet. 


5 January 2009 


The Department provided updated information to 
producers on program funding, but it was unable, at 
that time in program development, to specify details on 
the terms and conditions to participate. 


An information session with program details was 


held for tobacco producers. 


18 February 2009 


Producers asked a number of questions about their 
rights as TTP recipients and the rights of family 
members and relatives to obtain a licence. They also 
asked about transferring their quota to others. The 
Department and Tobacco Marketing Board presentation 
materials did not deal with quota transfers to non-quota 
holders. 


The Department issued a fact sheet to producers 


to answer questions raised at the information 
session. 


27 February 2009 


The fact sheet indicated that the spouse and dependent 
children of a TTP recipient would not be entitled to a 
licence. However, an independent child (a child over 18 
years of age) would be allowed to obtain a licence and 
sell tobacco. As well, a TTP recipient could become an 
employee of another tobacco farm if he/she did not have 
any direct or indirect interest in the ownership of that 
farm. The fact sheet was silent on quota transfers. 


The Department received complaints about quota 


transfers. 


3 March 2009 


The Department became aware that a large number of 
quota transfers to non-quota holders were being 
exercised. It received complaints that these quota 
transfers were being done to bypass the TTP conditions. 


The Department issued a letter to the Tobacco 
Marketing Board. 


25 March 2009 


In response to signs that tobacco producers were 
transferring quota to non-farming family members, the 
Department directed the Tobacco Marketing Board to 
scrutinize all quota transfers and to deny a licence to 
anyone whose actions could have the effect of 
undermining the purpose and intent of the transition 
program. 


Quota transfers were reversed. 


| 
25-30 March 2009 


The Department had to extend the date of application to 
the TTP program to 30 March 2009, to allow for 
reversal of the quota transfers. The number of quota 
transfers was reduced from 336 to 22. 
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Exhibit 3.2 Agriculture and Agri-Food Canada did not adequately prepare for the risks that might affect a program such as the Tobacco 
Transition Program (continued) 


The Department issued a letter to the Tobacco 9 April 2009 The letter was designed to direct the Tobacco Marketing 
Marketing Board. Board to curtail close business relationships between 
TTP recipients and their relatives. This included, for 
example, TTP recipients working on their own farm as 
employees and children renting their parents’ land and 
infrastructure and obtaining a licence to grow tobacco, 
when one or more parents was a TTP recipient. The 
letter was widely distributed to producers. 


The Department issued a letter to the Tobacco 7 May 2009 ire The Department changed its position on relationships 
Marketing Board. outlined in the letter of 9 April 2009. It communicated 
to the Tobacco Marketing Board that such arrangements 
were allowed as long as the business relationship was 
at arm’s length and, for example, rent or wages were at 
fair market value. 


The Department received letters asking for clarity January to 7 Producers were still unclear on various relationship 
on relationship scenarios. May 2010 scenarios between TTP recipients and potential licence 
: holders. 
— Ee 


3.88 The external auditors noted a trend for TTP recipients to enter 
into business relationships with family members who have a licence for 
the production of tobacco. For example, they found that parents who 
were ITP recipients were leasing land and equipment to their non- 
dependent children, and adult children (over 18 years of age) who 
were ITP recipients were leasing land and equipment to their parents. 
A variety of other business relationships existed, such as between 
brothers, sisters, grandparents, and in-laws, all of which met the terms 
and conditions of the program under the Funding Agreement. 


3.89 With respect to achievement of program objectives, the external 
audit concluded the following: 


* The objective to eliminate the quota system had been achieved. 


¢ The objective to improve the viability of remaining and future 
tobacco producers had been achieved (given that tobacco 
production more than doubled in 2010). 


* The objective to facilitate the transition of tobacco producers out of 
the tobacco industry was not as far advanced. Although TTP 
recipients were producing other crops, this production represented 
a small portion of their land. Recipients had also sought non-farm 
income, including renting their land and equipment to licensees to 


grow tobacco or becoming tobacco farm employees. 
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3.90 Lessons learned. The Department carried out a lessons-learned 
analysis of the program to identify the strengths and weaknesses of its 
design and implementation. Strengths noted included the following: 


* The Department obtained the appropriate approvals and money 
transfers in time. 


* The Department was responsive to producers’ calls and contacted 
them to clarify rules. 


Weaknesses noted included the following: 
¢ Producers did not clearly understand the purpose of the program. 


¢ Communication was unclear to the Tobacco Marketing Board 
about its role and how the program should be implemented. 


¢ The Funding Agreement should have covered quota transfers. 
¢ Greater attention was needed to anticipate issues that might arise. 


¢ The Tobacco Marketing Board is a regulatory body of the 
Province of Ontario and represents the interests of all Ontario 
flue-cured tobacco producers. For the purposes of the TTP this 
could be seen as being a conflict of interest. 


3.91 Recommendation: Agriculture and Agri-Food Canada should | 


* ensure an appropriate and timely follow-up on the findings of the 
Department’s external audits of the Tobacco Transition Program 
(TTP), where the auditors raised concerns about business 
relationships or where producers did not provide the requested 
information; and 


¢ develop a cost-effective strategy for future monitoring of business 
arrangements between TTP recipients and licensees, to ensure 
their appropriateness. 


The Department’s response. Agreed. The Department has written to 
the Ontario Flue-Cured Tobacco Grower’s Marketing Board to share 
the results of the audit findings and to set out expected next steps. 
Further to this follow-up, the Department will also develop a strategy 
for ongoing monitoring, on a cost-effective basis, of business 
arrangements between TTP recipients and licensees. This 
recommendation will be fully implemented by March 2012. 
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Conclusion 


3.92 Our overall audit objective was to determine whether 
Agriculture and Agri-Food Canada followed a risk-based approach in 
developing and implementing its payments to producers programs. 


3.93 We examined a number of initiatives carried out by the 
Department to improve the design of its producer income support 
programs, including industry consultation, strategic review, producer 
surveys, and focus groups. The Department, working with provinces 
and territories, has made progress on some design issues but long- 
standing concerns with AgriStability remain. These include unclear 
program objectives, lack of timely access to program funding, and 
program complexity that affects producers’ ability to reasonably predict 
payment amounts. Department officials indicated that they are aware 
of these issues and are discussing their implications during their 
consultation with industry and provincial/territorial partners for 
Growing Forward 2 programs. 


3.94. Implementation of Agrilnvest through the financial institutions 
was delayed. For each of the first three program years of a five-year 
program, producers had a different process for participating in 
Agrilnvest. Despite the delay in implementing the program through 
financial institutions, producers benefited. They received $563 million 
in Agrilnvest Kickstart funds with no matching deposit required. As 
well, for the 2007 program year, the Department waived the producer 
deposit requirement. 


3.95 With respect to program administration, the Department has 
made progress in addressing issues raised in our 2007 audit, such as 
better managing underpayments to producers and payment accuracy. 
However, the Department has not systematically followed up on the 
causes of delays in payments to producers to accelerate payments. 
Although processing times for AgriStability payments have improved, 
the Department has not met its targets, and producers can wait up to 
two years to receive a payment subsequent to an income loss. In 
addition, the Department has not collected information on processing 


time for Agrilnvest. 


3.96 We found that the Department had carried out a risk assessment 
and identified risk mitigation strategies for Agrilnvest and 
AgriStability. However, it needed to specify risk management activities 


and timelines and periodically monitor progress on activities. 
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3.97 While federal/provincial/territorial accountabilities for 
performance reporting were not specified in the Growing Forward 
Framework Agreement, a performance measurement framework has 
since been agreed to by all governments. The provinces now deliver 
88 percent of AgriStability payments. Full implementation of the 
performance measurement framework is needed to provide an overall 
picture of performance against service standards to Parliament and the 
public. In addition, we found that the Department followed a sound 
process in assessing provincial capacity and transferring AgriStability 
administration to the provinces of British Columbia and Saskatchewan 
in 2010. However, it was not clear whether the federal government or 
the provinces would monitor and report on business case 
commitments. 


3.98 The Department had to develop the Tobacco Transition Program 
within a short time frame and did not first conduct a thorough risk 
analysis. The agreement implementing the program did not provide 
clear terms and conditions to ensure that recipients would not enter 
into business arrangements that would undermine the intent of the 
program. The Department changed its interpretation of what was and 
was not allowed under the agreement a number of times, resulting in 
confusion for producers. As a result, the Department was successful in 
controlling some, but not all, business arrangements that it believed 
would undermine the intent of the program. 


3.99 For AgriStability and Agrilnvest, we concluded that Agriculture 
and Agri-Food Canada followed a risk-based approach in developing 
and implementing its payments to producers programs, except for some 
needed improvements to analysis and follow-up and to public reporting 
on performance. 


3.100 For the Tobacco Transition Program, we concluded that 
Agriculture and Agri-Food Canada did not follow a risk-based 
approach in developing and implementing the program. 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants (CICA). While the Office adopts 
these standards as the minimum requirement for our audits, we also draw upon the standards and practices 
of other disciplines. 


Objectives 


The overall audit objective was to determine whether Agriculture and Agri-Food Canada followed a 
risk-based approach in developing and implementing its payments to producers programs. 


There were two sub-objectives; these were to determine whether Agriculture and Agri-Food Canada 
- followed a risk-based, lessons-learned approach in developing and improving the programs; and 


- established clear accountability and monitoring practices in implementing the programs. 


Scope and approach 


The scope of the audit includes three payments to producers programs: Agrilnvest, AgriStability, and the 
Tobacco Transition Program. It also includes the process of transferring administration of AgriStability to 
Saskatchewan and British Columbia. 


The audit focused on 
+ the process followed to develop and improve the programs; 


+ the process followed for transferring administration of AgriStability to Saskatchewan and British 
Columbia; and 


- the Department’s accountability and monitoring in implementing the programs, including follow-up of 
our May 2007 chapter that covered the Canadian Agricultural Income Stabilization (CAIS) program. 


The Department’s internal auditors carried out some of the work on behalf of the Office of the Auditor 
General team. Specifically, they followed up on recommendations in our 2007 CAIS chapter, as they 
pertained to AgriStability. In addition, external auditors for the Department conducted follow-up work on 
the Tobacco Transition Program, as reported in our chapter. We took the steps required by CICA standards 


for reliance. 


During the audit, we collected information through analysis of files, records, reports, and bilateral 
agreements. We also conducted interviews with the Department’s management and staff responsible for 
the design and delivery of these programs, as well as interviews with third-party delivery organizations (that 
is, provincial government agencies, financial institutions, and other delivery partners) and stakeholders. 
The audit included trips to the Department's Farm Income Programs Directorate, located in Winnipeg. 


We met with third-party delivery partners, stakeholders, and provincial government agencies; however, we 


did not audit their work. 
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Criteria 


The criteria used in this audit apply to the time period in which the programs were delivered. 


Agriculture and Agri-Food Canada identifies risks specific to each | * Integrated Risk Management Framework, 
program and the measures that will be used to manage these Treasury Board, 2001 
risks in a timely manner. 


The Department identifies and considers lessons learned in ¢ Integrated Risk Management Framework, 
planning the new program. i Treasury Board, 2001 


Where third-party delivery or provincial delivery was involved, Directive on Transfer Payments, Treasury Board, 2008 
the Department ensures that the delivery organization was 


suitable to deliver the program. 


Policy on Transfer Payments, Treasury Board, 2000 


The Department uses input provided by partners, producers, and | * Communications Policy of the Government of Canada, 
stakeholders to identify gaps in communication and to improve Treasury Board, 2006 
program design and operations. ; 


Integrated Risk Management Framework, 
Treasury Board, 2001 


Program delivery and/or funding agreements between the * Directive on Transfer Payments, Treasury Board, 2008 
Department and delivery organizations or provinces include 
written accountabilities that are monitored. 


Policy on Transfer Payments, Treasury Board, 2000 


The Department has systems in place aimed at continuous ¢ A Policy Framework for Service Improvement in the 
improvement. Government of Canada, Treasury Board, 2000 


Integrated Risk Management Framework, 
Treasury Board, 2001 


Management reviewed and accepted the suitability of the criteria used in the audit. 


Period covered by the audit 


The audit covered both the development and implementation of the programs. For Agrilnvest and 
AgriStability, the audit covered the period between February 2006 and May 2011. For the Tobacco 
Transition Program, the audit covered the period between May 2004 and May 2011. Audit work for this 
chapter was substantially completed on 2 May 2011. 


Audit team 


Assistant Auditor General: Neil Maxwell 
Principal: Katherine Rossetti 
Director: Dawn Campbell 


Alina Dan 
Erika Boch 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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Appendix List of recommendations 


The following is a list of recommendations found in Chapter 3. The number in front of the 
recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses 
indicate the paragraphs where the topic is discussed. 


Improving program design 


3.28 Agriculture and Agri-Food 
Canada should work with provinces 
and territories to help increase 
producers’ and stakeholders’ 
understanding of the objectives and 
trade-offs of income support programs 
and to clarify government’s and 
producers’ responsibilities for managing 


farm risks. (3.11-3.27) 


3.29. Agriculture and Agri-Food 
Canada should work with service 
delivery partners to understand their 
challenges and develop and monitor 
realistic implementation timelines for 


future income support programs. 
(3.11-3.27) 
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Agreed. The current suite of programs was designed with 
industry to respond to the needs of producers. In conjunction 
with provincial and territorial governments, the Department 
continues to consult extensively with industry in the 
development of programming, most recently through two rounds 
of industry engagement on the next generation of the 
agricultural policy framework (Growing Forward 2). Through 
future rounds of broad-based engagement, as well as more 
targeted sessions with producers, their associations, and formal 
mechanisms such as through the National Programs Advisory 
Committee, the Department will continue to gain a common 
understanding with producers of the respective roles in 
managing risk, of the objectives of programming, and of the 
inherent trade-offs within programs. This recommendation will 
be fully implemented with Growing Forward 2 in April 2013. 


Agreed. In delivering programs, the Department’s objective is to 
ensure that the intended benefits of programming are available 
to participants in a timely manner. Under the Growing Forward 
2 policy framework, the Department will actively engage service 
delivery partners in the process of developing and implementing 
income support programs. Officials will work with these partners 
to ensure that challenges are identified and documented, and 
that workable and timely solutions are developed to secure 
appropriate service delivery for producers. As it did with the 
delivery of Agrilnvest, the Department will also ensure that 
contingency plans are in place so that producers have access to 
program benefits despite challenges that may arise with the 
intended delivery arrangement. By relying on such contingency 
plans, the Department was able to ensure that producers had 
access to Agrilnvest funding, although the infrastructure with 
financial institutions was not in place for the 2007 and 2008 
program years. This recommendation will be fully implemented 
with Growing Forward 2 by April 2013. 
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Improving program administration 


3.40 Agriculture and Agri-Food 
Canada should develop and implement 
formal tracking and reporting systems 
to ensure that key planned actions for 
AgriStability and Agrilnvest are carried 
out as expected. (3.35-3.39) 


3.46 Agriculture and Agri-Food 
Canada should analyze processing of 
payments for AgriStability and 
Agrilnvest in a more systematic manner 
and follow up on remedies to improve 
the timeliness of payments to 


producers. (3.41-3.45) 


Agreed. As a matter of practice, we work with provinces and 
territories to monitor, assess, and evaluate programs with a view 
to continuous improvement. The development of farm income 
support programs under the current Growing Forward 
Framework Agreement was informed by audits, evaluation, 
reviews, and assessments. As this type of work is undertaken for 
the next framework agreement, officials will implement a formal 
system to track documents to demonstrate the actions taken to 
address the recommendations, proposed actions, and risk- 
mitigating strategies that are carried out. This recommendation 
will be fully implemented with Growing Forward 2 by 

April 2013. 


Agreed. The Department is aware of the concerns about the 
timeliness of program payments. Leading up to the 
implementation of the current suite of programs, the 
Department reviewed potential approaches to improving the 
timeliness of payments in a number of venues with industry, as 
well as with provincial and territorial governments. The 
Department will more systematically review the design of 
income support programming as part of Growing Forward 2 
discussions and will analyze its payment processes to improve the 
timeliness of processing of payments. In that regard, the recent 
implementation of an updated application processing system will 
provide the Department with the capacity to produce more 
systematic reporting on application processing. The Department 
will use the reporting capacity to further improve the timeliness 
of producer payments. This recommendation will be fully 
implemented with Growing Forward 2 by April 2013. 


Monitoring federal/provincial delivery 


3.55 Agriculture and Agri-Food 
Canada should add, to its future 
business case requirements, the need for 
a proposal from the provinces on 
whether the federal government or 
provincial governments will monitor 


and report on commitments in the 
business case. (3.47-3.54) 


Agreed. The Department will clarify how the monitoring and 
reporting of key provincial commitments (not already covered by 
existing mechanisms) should be addressed in any future business 
cases. 
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3.66 Agriculture and Agri-Food 
Canada should work with the provinces 
to finalize and fully implement the 
agreed-upon performance measurement 
framework and to improve the 
completeness and timeliness of 
reporting to Parliament and the public 
on income support programs. 


(3.59-3.65) 


Agreed. The Department is committed to measuring and 
reporting on program performance and has developed an income 
support programs measurement framework with the provinces 
and territories to do so. The Department will refine the 


framework and leverage it in its departmental performance 


reporting so that the reports are consistent and provide 
Parliament and the public with a complete view of program 


performance. This recommendation will be fully implemented 
with Growing Forward 2 by April 2013. 


—_—_—_—_—eeee eee ee 


Managing the Tobacco Transition Program 


3.82 For future one-time payment 
programs, Agriculture and Agri-Food 
Canada should conduct a thorough risk 
assessment during program 
development and maintain a risk 
register to support corrections 
throughout program delivery. 
(3.67-3.81) 


3.91. Agriculture and Agri-Food 
Canada should 


* ensure an appropriate and timely 
follow-up on the findings of the 
Department’s external audits of the 
Tobacco Transition Program (TTP), 
where the auditors raised concerns 
about business relationships or where 
producers did not provide the 
requested information; and 


develop a cost-effective strategy for 
future monitoring of business 
arrangements between TTP 
recipients and licensees, to ensure 
their appropriateness. 


(3.83-3.90) 
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Agreed. The Department recognizes the need to conduct 
thorough risk analysis and develop mitigation strategies 
commensurate with program risks when implementing one-time 
payment programs. Consistent with the Policy on Transfer 
Payments, the Department is adopting a risk-based management 
approach to program design and implementation. Additional 
guidance and tools are under development to guide the 
development of program risk registers and ongoing risk 
management practices. This recommendation will be fully 
implemented by December 2012. 


Agreed. The Department has written to the Ontario Flue-Cured 
Tobacco Grower’s Marketing Board to share the results of the 
audit findings and to set out expected next steps. Further to this 
follow-up, the Department will also develop a strategy for 
ongoing monitoring, on a cost-effective basis, of business 
arrangements between TTP recipients and licensees. This 
recommendation will be fully implemented by March 2012. 
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Performance audit reports 


This report presents the results of a performance audit conducted by the Office of 
the Auditor General of Canada under the authority of the Auditor General Act. 


A performance audit is an independent, objective, and systematic assessment 

of how well government is managing its activities, responsibilities, and resources. 
Audit topics are selected based on their significance. While the Office may 
comment on policy implementation in a performance audit, it does not comment 
on the merits of a policy. 


Performance audits are planned, performed, and reported in accordance with 
professional auditing standards and Office policies. They are conducted by 
qualified auditors who 


* establish audit objectives and criteria for the assessment of performance; 
« gather the evidence necessary to assess performance against the criteria; 
* report both positive and negative findings; 

* conclude against the established audit objectives; and 


* make recommendations for improvement when there are significant 
differences between criteria and assessed performance. 


Performance audits contribute to a public service that is ethical and effective 
and a government that is accountable to Parliament and Canadians. 


Table of Contents 


Main Points 


Introduction 


Regulating pharmaceutical drugs in Canada 
Stakeholders in pharmaceutical drug safety 
Health Canada’s regulatory approach 

Focus of the audit 


Observations and Recommendations 


Regulating clinical trials 

Clinical trial applications and amendments were reviewed in a timely manner 

Additional steps are needed to strengthen a risk-based approach to oversee Clinical trials 
Authorized clinical trials were not disclosed publicly 

Reviewing drug submissions 


Health Canada does not meet its own service standards for reviewing drug submissions 


Health Canada has not assessed whether its review bureaus interpret and apply review procedures 
and guidelines consistently 


More information on Health Canada’s reviews of drug submissions needs to be made 
available to Canadians 


Health Canada has not determined what measures are necessary to manage conflict-of-interest risks 
for drug reviewers 


Monitoring post-market safety 


Health Canada has recently taken additional steps to actively monitor drug safety 
Health Canada’s assessment of, and response to, potential safety issues is not timely 


Enforcing compliance with the regulations 


Actions have been taken to make inspections of drug establishments more risk-based 
Complaints about marketed drugs are not prioritized consistently 


Conclusion 
About the Audit 


Appendix 


List of recommendations 
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Main Points 


Pharmaceutical drugs are mostly synthetic products made from 
chemicals. They are meant to improve the health and well-being of 
patients by helping to prevent and treat disease, reduce pain and 
suffering, and extend and save lives. Some higher-risk drugs, such as 
those used to treat diseases, require a prescription from a physician. 
Other lower-risk drugs, such as cough syrup and antacids, are sold 
without a prescription and are readily available to the public. 


Health Canada, through the Food and Drugs Act, regulates the safety, 
efficacy, and quality of all pharmaceutical drugs for use by humans in 
Canada before and after the products enter the Canadian marketplace. 
The Department does this through a combination of scientific review, 
monitoring, compliance, and enforcement activities. It aims to ensure 
that the public has timely access to safe and effective pharmaceutical 
drugs and that those who need to know of safety concerns are informed. 


We examined how Health Canada regulates clinical trials of new 
pharmaceutical drugs and reviews submissions seeking approval of new 
drugs for sale in Canada or of changes to drugs already on the market. 
We also examined how the Department monitors product safety and 
ensures that potential safety concerns are communicated to health 
care professionals and the public. In addition, we looked at how Health 
Canada enforces industry compliance with regulatory requirements 
governing the testing, production, and sale of drugs. We did not 
examine the soundness of the Department’s regulatory decisions or the 
safety or efficacy of drugs. 


The period under audit for this chapter was | January 2009 to 
31 December 2010. Audit work for this chapter was substantially 
completed on 31 May 2011. 


There are about 13,000 prescription and non-prescription drugs on 
the Canadian market. Pharmaceutical drugs play an important role 
in Canada’s health care system and economy. In 2008, the Canadian 
retail market for prescription and over-the-counter drugs was valued 


at about $28 billion, with prescription drug purchases accounting for 
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What we found 


almost 84 percent of total retail drug expenditures. According to IMS 
Brogan, a well-recognized provider of data to Health Canada and the 
pharmaceutical industry, about 505 million prescriptions were 
dispensed by Canadian retail pharmacies in 2010. 


With an aging population, the role of pharmaceutical drugs is expected 
to grow as researchers come up with new drug therapies to replace 
earlier treatments or provide new options where no treatment existed 
before. Canadians who purchase and consume pharmaceuticals 
authorized for sale in Canada rely on the government and industry to 
monitor the safety of these products. Health Canada has a 
responsibility to help protect the public against undue health and 
safety risks from the use of pharmaceutical drugs. 


> The Department does not take timely action in its regulatory 
activities, with the exception of its review of two types of drug 
submissions. In particular, the Department is slow to assess potential 
safety issues. It can take more than two years to complete an 
assessment of potential safety issues and to provide Canadians with 
new safety information. 


- The Department received 4,400 drug submissions in 2009 and 2010. 
It has put in place processes and procedures to ensure that its drug 
reviews are consistent and high quality. However, it has not assessed 
whether these processes and procedures have been consistently 
interpreted and applied across its four review bureaus. 


* Health Canada does not disclose information on drug submissions 
that it has rejected or information on the status of the drugs it has 
approved with conditions. In addition, the Department has not 
acted on its long-standing commitment to disclose more information 
about clinical trials it has authorized. This increases the risk that 
Canadians may be unaware of new treatment options or may 
unknowingly participate in an unauthorized trial. 


* Health Canada’s conflict-of-interest guidelines and Code of Conduct 
are consistent with government policy on conflict of interest. 
However, unlike another major regulator of pharmaceutical drugs and 
some federal departments that have developed conflict-of interest 
requirements for specific work assignments, the Department has not 
determined what measures are necessary for its review activities. 


The Department has responded. The Department agrees with 
all of our recommendations. Its detailed responses follow the 
recommendations throughout the chapter. 
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Introduction 


Regulating pharmaceutical drugs in Canada 


4.1 Each day, Canadians and their health care providers use 
pharmaceutical drugs (herein referred to as drugs) that have been 
approved by Health Canada to treat or prevent an array of diseases and 
disabling physical conditions. Enabling timely access to safe and effective 
drugs, and ensuring that these products remain safe and effective, is 
critical to improving and maintaining the health of Canadians. 


4.2 Drugs are regulated under the Food and Drugs Act, which 

is administered by Health Canada. The Department defines 
pharmaceutical drugs as synthetic products made from chemicals, 
including 


* prescription and non-prescription drugs; 
* disinfectants; and 


* products, such as sunscreens and antiperspirants, 
that are usually low risk. 


4.3 There are about 13,000 drugs on the Canadian market, many of 
which are critical to high-quality health care. Canadians were 
expected to spend about $31 billion on these drugs in 2010. According 
to IMS Brogan, a well-recognized provider of data to Health Canada 
and the pharmaceutical industry, about 505 million prescriptions were 
dispensed by Canadian retail pharmacies in 2010. 


4.4 Under the Food and Drugs Act and its accompanying regulations, 
Health Canada, as federal regulator, is responsible for assessing and 
monitoring the safety and efficacy of drugs marketed in Canada. The 
Department carries out these responsibilities through various regulatory 
activities that are designed to evaluate and monitor the safety, efficacy, 
and quality of drugs before and after they are marketed. Some of the 
costs of these activities are shared with industry, but others, such as 
reviewing clinical trial applications and compliance and enforcement 
activities for clinical trials, are funded solely by the Department. In the 
2009-10 fiscal year, Health Canada spent about $80 million in direct 
program costs and employed approximately 700 full-time employees for 
drug regulation. The Department received about $33 million in fees 


from the pharmaceutical industry during the same period. 
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Stakeholders in pharmaceutical drug safety 


45 There are several important stakeholders whose participation in 
the regulatory system is critical to supporting the safe and effective use 
of drugs. 


46 The Canadian public. Canadians rely on Health Canada to 
approve drugs of demonstrated safety, quality, and efficacy and to work 
with the pharmaceutical industry to disseminate safety and usage 
information that is accurate and up to date. Canadian volunteers 
participating in clinical trials also rely on the Department to verify that 
authorized trials are designed appropriately, so they are not exposed to 
unnecessary risks. 


47 Health care professionals. Health care professionals play an 
important role in promoting the appropriate use of drugs. To provide 
the public with information on the risks and benefits of specific drugs, 
health care professionals rely on Health Canada and drug 
manufacturers to disseminate safety and usage information that has 
been officially approved and that is accurate and up to date. 


48 Health care professionals can also play a role in monitoring the 
continued safety of approved drugs. They are often the first to become 
aware of serious adverse drug reactions, and this makes them a critical 
source of safety information. Health Canada asks them to pass on this 
information when it comes to their attention, either to the 
manufacturer or directly to the Department. By doing so, they 
contribute to the evolving knowledge of a product’s risks and benefits. 


49 Pharmaceutical industry. The legal responsibilities of the 
pharmaceutical industry are outlined in the Food and Drug Regulations. 
The regulations place significant responsibility on the industry to 
protect the health and safety of the public. For example, the 
pharmaceutical industry is required to obtain authorization from 
Health Canada before conducting clinical trials of unapproved drugs. 
In its drug submissions to the Department, the industry is required to 
provide evidence that a drug meets safety, efficacy, and quality 
requirements. After the Department has approved a drug for sale, 
manufacturers are required to report all serious adverse drug reactions 
to Health Canada and to maintain the quality of their product. 
Manufacturers are also responsible for communicating new safety 
information about their products to health care professionals and 
consumers. 
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Health Canada’s regulatory approach 


4.10 Health Canada’s approach to the regulation of drugs focuses on 
well-defined points in the regulatory process that lead to a drug’s 
marketing approval. However, after the Department has authorized a 
drug for sale, it has limited regulatory authority to require label 
changes that address new safety information or to require 
manufacturers to undertake additional post-market studies. Other 
regulators have greater post-market regulatory authorities. For 
example, the US Food and Drug Administration can legally require 
industry to propose labelling changes to reflect new safety information 
within 30 days of a request. 


4.11 Health Canada’s responsibilities include the following 
core activities: 


* reviewing clinical trial applications, for clinical trials to be 
conducted in Canada; 


* reviewing drug submissions from manufacturers for market 
authorization and for post-market changes; 


* monitoring the safety of drugs in the Canadian market and 
communicating safety risks to health care professionals and the 
public, in collaboration with industry; 


* enforcing the pharmaceutical industry’s compliance with 
regulations, including those related to clinical trials, drug 
manufacturing, and the reporting of adverse drug reactions. 


Exhibit 4.1 includes an overview of the regulatory process. 


Exhibit 4.1 Regulatory process for drugs in Canada 


Pre-Market Post-Market 


Regulato Market Survellanca, 
Pre-clinical Clinical waved Submission ; Public inspection, 
A ; product : authorization 
studies trials ayes Behe review Seriaen access and 
: investigation 
Source: Health Canada 
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Regulating clinical trials 


Focus of the audit 


4.12 The focus of our audit was to determine whether Health Canada 
fulfilled its key responsibilities related to clinical trials, submission 
reviews, and post-market activities for the regulation of drugs. 


4.13 We examined whether the reviews of clinical trial applications 
and pharmaceutical submissions were timely and whether the 
Department had established processes to support the timely, consistent, 
and high-quality review of drug submissions. We examined the steps the 
Department took to support the transparency of authorized clinical 
trials and its reviews, and the systems it implemented to manage 
potential conflicts of interest. We looked at how the Department 
monitored the safety of drugs in clinical trials and of drugs that had 
already been marketed, and whether it communicated safety concerns 
to health care professionals and Canadians in a timely manner. Finally, 
we examined the Department's methods for verifying the regulatory 
compliance of the pharmaceutical industry. 


4.14 We did not examine authorized clinical trials to determine 
whether they were safe for the participants. We did not examine 
completed reviews of drug submissions to determine whether drugs 
approved by the Department were safe or effective or whether they 
were reviewed in a consistent manner. We did not examine the risks 
and benefits of marketed drugs. 


4.15 More details about the audit objectives, scope, approach, and 
criteria are in About the Audit at the end of this chapter. 


Observations and Recommendations 


4.16 Clinical trials are experiments involving volunteer participants 
that are used to determine whether a drug is safe and effective and 
what side effects are associated with its use. Parties seeking to conduct 
clinical trials in Canada must submit a clinical trial application to 
Health Canada, except for clinical trials on drugs that are already 

on the market and are being tested to treat conditions for which 

they were authorized. 


4.17 In 2001, the government revised the Food and Drug Regulations to 
strengthen protection for clinical trial participants. Health Canada also 
developed a national inspection program to verify that clinical trials 
conducted in Canada comply with these regulations, which were 
designed to protect the participant’s safety and to generate high-quality 
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clinical data. The Department estimates that there are 4,000 active 
clinical trial sites (many sites are testing the same drug) in the country 
each year. If its inspectors identify significant non-compliance with the 
regulations, the Department can require immediate corrective action 
or, if necessary, revoke the trial’s authorization. 


Clinical trial applications and amendments were reviewed in a timely manner 


418 In 2009 and 2010, Health Canada reviewed about 

2,600 applications, and about 1,800 amendments to clinical trials, 

to assess whether the proposed trial or amendment posed undue risks 
to trial participants. Under the Food and Drug Regulations, if Health 
Canada does not review applications and amendments within 30 days, 
the trial can proceed or the amendment can be implemented by default. 
In addition, for selected clinical trial applications and amendments, the 
Department established a 7-day accelerated review target. 


4.19 ‘To determine the timeliness of its reviews of clinical trial 
applications and amendments in 2009 and 2010, we examined 
whether the Department was meeting the 30-day and 7-day timelines. 
We reviewed Health Canada’s performance reports and assessed the 
completeness and accuracy of the data used to create these reports. 
We also examined management reports on program performance for the 
period under audit, analyzed data, and interviewed key officials. We 
found that the reviews of clinical trial applications and amendments 
were timely. The Department made all of its review decisions within the 
required 30-day period and met the 7-day target for selected clinical 
trial applications and amendments 90 percent of the time. 


Additional steps are needed to strengthen a risk-based approach to oversee 
clinical trials 


4.20 Monitoring adverse drug reactions. We examined whether 
Health Canada had established a risk-based approach to monitor 
adverse drug reactions in clinical trials. The Food and Drug Regulations 
require that clinical trial sponsors, such as drug companies and 
hospitals, inform the Department of all serious, unexpected adverse 
drug reactions for drugs being tested in Canadian clinical trials— 
regardless of whether the adverse drug reaction occurred at a trial site 
in Canada or in another country. The number of reported adverse drug 
reactions in clinical trials has increased dramatically over the past 
several years—43,000 in 2007, 88,000 in 2009, and 115,000 in 2010. 
According to the Department, about 95 percent of these reports are 


from foreign sources. 
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4.21 Monitoring and assessing adverse drug reactions is important for 
ensuring that Canadians participating in clinical trials are protected 
and fully informed of potential safety risks. Monitoring the safety of the 
700 drugs that are tested in clinical trials in Canada each year also 
provides the Department with information that can be used by drug 
reviewers, clinical trial inspectors, and officials responsible for 
monitoring the safety of marketed drugs. 


4.22. We reviewed the Department’s procedures to determine whether 
it had established a risk-based approach for monitoring and assessing 
adverse drug reaction reports. 


4.23 We found that Health Canada receives all adverse drug reaction 
reports by fax or courier and manually enters them into its adverse drug 
reaction database for clinical trials. The process is labour intensive, and 
it uses resources that could otherwise be used to assess potential safety 
issues raised in these reports. A description of how the Department 
expects to address this issue can be found in paragraph 4.75. 


4.24 Due to the significant number of drugs in clinical trials each year, 
the volume of reports received, and the labour-intensive process in 
place, it is important that Health Canada have a risk-based approach 
to monitoring drugs in these trials. Officials told us that assessment 
officers were monitoring drug reaction reports from the trials that 
pose the highest risks, such as early-phase clinical trials of drugs not 
previously tested and trials that include vulnerable populations (for 
example, children). However, at the time of our audit, there were no 
standard operating procedures for its monitoring activities to ensure 
that the Department consistently focused on the trials that posed the 
greatest risk. 


4.25 When the Department receives adverse drug reaction reports 
that indicate there may be a safety issue with a drug, officials may 
choose to fully assess the adverse reaction and its relationship to the 
drug. The assessment may result in recommendations being issued to 
ensure that risks are communicated to and, if possible, reduced for 
clinical trial participants. While officials told us that hundreds of 
potential safety issues are awaiting assessment, we found that, at the 
time of our audit, the Department had not documented its criteria for 
prioritizing its assessment of these potential safety issues based on the 
tisks posed to clinical trial participants. 


4.26 Inspecting clinical trial sites. We examined Health Canada’s 
approach to clinical trial inspections to determine whether its 
compliance and enforcement activities focused on those trials that 
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posed the greatest risk. The Department inspects Canadian clinical 
trial sites to verify that authorized trials comply with the Food and Drug 
Regulations, so the rights and safety of trial subjects are respected. It 
also verifies that the data generated by the trial site is of high quality, 
which is important because this data can be used to support 
submissions for new drugs seeking market authorization. The 

June 2010 draft Compliance and Enforcement Risk Evaluation Guide: 
An Approach to Decision Making, by the Health Product and Food 
Branch Inspectorate, specifies that compliance and enforcement 
activities associated with a regulated product or activity need to be 
appropriate and proportional to the risk posed. 


4.21 ‘To determine whether compliance and enforcement activities for 
clinical trials were risk-based, we reviewed the Department’s clinical 
trial inspection strategy, its performance reports, and its procedures for 
inspections. We also reviewed finalized inspection reports for the six 
non-compliant clinical trials identified in 2009 and 2010. We assessed 
whether the Department obtained assurance that instances of non- 
compliance were addressed. 


4.28 We found that Health Canada had developed a risk-based 
inspection strategy that included criteria to help inspectors determine 
which clinical trial sites to inspect. The strategy required that inspectors 
consider a number of potential risk factors, including the following: 


* number of clinical trials conducted at the site, 
* number of subjects enrolled in the specified clinical trial, 


* number of serious unexpected adverse drug reactions at the 
clinical trial site, and 


* observations made during past inspections. 


4.29. However, we also found that Health Canada does not regularly 
collect all of the information necessary to assess these factors and to 
make comparative risk-based decisions. Because clinical trial sponsors 
are not required to submit up-to-date information on clinical trial 
sites, inspectors must call each site directly to find out the current 


status of the clinical trial site and the number of participants enrolled. 


4.30 Officials told us that acquiring this information through direct 
contact with each clinical trial site is inefficient and that a significant 
amount of time is devoted to identifying potential inspection sites. 
Thus, inspectors have up-to-date information only for sites that they 


call and are unable to compare the risks posed by all sites. 
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4.31 Health Canada’s strategy is to inspect two percent (that is, 
about 80 out of 4,000) of Canadian clinical trial sites in any given year. 
The Department told us that this target is consistent with the 
approaches taken by other major regulators. We found that the 
Department completed 52 inspections in 2009 and 50 in 2010. 
Officials indicated that the target (of 80 per year) could not be met 
because of a lack of resources and the reallocation of existing resources 
to other programs. 


4.32 Since 2006, Health Canada has issued nine inspection reports 
with non-compliance ratings—six of which were issued in 2009 and 
2010. We reviewed these six reports to determine whether the 
Department verified that instances of non-compliance with the Food 
and Drug Regulations were addressed. Depending on the nature of 
identified deficiencies, corrective actions may be required to protect 
the safety of clinical trial participants and to ensure the quality of 
clinical trial data. 


4.33 For these six reports, we found that Health Canada took between 
56 and 142 days to officially notify regulated parties that they were not 
compliant with the Food and Drug Regulations and to officially request 
corrective actions to address all identified deficiencies. The Department 
has not established timelines for issuing these notifications, but 
regulated parties are required to propose corrective actions within 

four weeks of receiving an official notification of non-compliance. 


4.34 Health Canada also reviews the adequacy of corrective actions 
proposed by regulated parties, but it has not set timelines for this 
review, either. During our audit, the Department reviewed proposed 
corrective measures for two non-compliant inspection reports and 
took about 110 days for each review. Officials told us that, during this 
time, the Department requested and reviewed additional information 
provided by regulated parties. 


4.35 Recommendation. Health Canada should strengthen its risk- 
based approach for monitoring and assessing clinical trial adverse drug 
reaction reports and for inspecting clinical trial sites, so potential safety 
issues are mitigated. 


The Department’s response. Agreed. The Department is 
strengthening its risk-based approach to monitor and assess clinical 
trial adverse drug reaction reports and clinical trial inspections. 

A detailed standard operating procedure and strategy guide has 
been developed to prioritize the review of individual adverse drug 
reaction reports. This approach was implemented on 4 July 2011. 
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The Department expects to have completed a review of the existing 
risk-based process for selecting clinical trial inspection sites by 

fall 2011. This review will be used to assess the effectiveness of the 
existing process and to inform the development, documentation, and 
implementation of an enhanced process. 


4.36 Recommendation. Health Canada should establish timelines for 
officially notifying clinical trial sites of non-compliant ratings and for 
reviewing proposed corrective measures to verify compliance with the 
Food and Drug Regulations. 


The Department’s response. Agreed. The Department is currently 
reviewing and revising its existing standard operating procedure for 
conducting clinical trial inspections. This revised standard operating 
procedure will emphasize establishing timelines for key steps in the 
inspection process, including notification of non-compliant ratings and 
the review of proposed corrective measures. This work will be 


completed by 31 March 2013. 


Authorized clinical trials were not disclosed publicly 


4.31 We examined steps taken by Health Canada to support the 
transparency of authorized clinical trials. In 2004, the House of 
Commons Standing Committee on Health recommended that the 
Department create a“... public database that provides information on 
trials in progress, trials abandoned and trials completed.” In its 

2007 Blueprint for Renewal II: Modernizing Canada’s Regulatory 
System for Health Products and Food, the Department committed to 
enhancing public access to clinical trial information. Without this 
information, Canadians with life-threatening diseases may not become 
aware of trials that could offer new treatment options, and may not be 
able to verify whether the Department has authorized advertised trials. 


4.38 We found that, despite commitments to increase the 
transparency of authorized clinical trials, the amount of information 
Health Canada made available to Canadians had not changed. There 
remains no definitive, publicly accessible source of information on 
clinical trials authorized by the Department. 


4.39 For example, in 2008, Health Canada became aware of an 
unauthorized clinical trial when it was contacted by parents whose child 
was enrolled in the trial and who had concerns about the safety of the 
drug being tested. The physician running the trial was not based in 
Canada but was recruiting Canadian participants. According to files 


compiled by Health Canada during its review of the complaint, 
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advertisements for the trial claimed that the Department had authorized 
the trial. It was not until the parents contacted the Department with 
their concerns that they learned that it had not, in fact, authorized the 
trial. Electronic access to a listing of trials authorized by the Department 
would allow Canadians to consult official information, to verify claims 
made by other parties, and to make fully informed decisions. 


4.40 In 1999 and 2004, parliamentary committees requested that 
Health Canada report annually on the findings of clinical trial 
inspections. Although openness and transparency are key aspects of its 
strategic and operational plans, the Department reported publicly only 
on its inspection activities in 2003 and 2004. For the past two years, it 
planned to publish a summary of its inspections on its website. At the 
time of our audit, a summary report on the clinical trial inspections 


conducted between 2002 and 2010 had been drafted but not published. 


441 Recommendation. Health Canada should fulfill long-standing 
commitments to enhance public access to information on authorized 
clinical trials, including the results of its clinical trial inspections. 


The Department’s response. Agreed. The Department will develop 
policies on enhancing public access to information on authorized 
clinical trials that respect privacy rights and legislation. Work began 
in 2011 and will be completed by 31 March 2013. 


The Department commits to publishing periodic reports regarding 
clinical trial inspections, to provide stakeholders and the public 
with a summary view of its inspection findings by 31 March 2012. 


Reviewing drug submissions 4.42 Health Canada received about 4,400 drug submissions in 2009 
and 2010. The Department reviewed pre-market submissions to 
determine whether claims made by industry regarding a drug’s safety, 
efficacy, and quality were supported by sufficient evidence. 


4.43 Submissions for new drugs that comprise chemicals that have not 
been available for a long period of time (typically, prescription drugs) 
must include a significant amount of data from clinical trials. 
Submissions for drugs that have been available for longer and have 
established safety records (typically, over-the-counter drugs) do not 
usually require as much data. 


4.44 Because the safety of each drug is not absolute, Health Canada 
must weigh the potential benefits and risks of those seeking access to 
the Canadian market to determine whether the risks are acceptable. 
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Post-market submissions are submitted when manufacturers wish to 
make changes to a label or manufacturing method for a product that 
has already been marketed. 


Health Canada does not meet its own service standards for reviewing drug 
submissions 


4.45 Health Canada reviews a variety of pre-market submissions to 
determine whether they include sufficient evidence to support the 
pharmaceutical industry’s claims about drug safety, efficacy, and quality. 
In consultation with the pharmaceutical industry, the Department 
developed service standards for these reviews in 1996 (Exhibit 4.2). 


4.46 To determine the timeliness of its review decisions, we examined 
whether the Department was meeting its own service standards for 
reviewing pharmaceutical submissions. We also examined management 
reports related to program performance for the period under audit, 
analyzed data, and interviewed key officials. 


4.47 We found that in 2009 and 2010, Health Canada consistently 
met its service standards for timely reviews of supplemental new drug 
submissions. However, it did not consistently meet its service standards 
for reviewing most of the other submission types. The Department's 
performance for reviewing generic drugs, over-the-counter drugs that 
required a clinical review, and post-market change submissions was 
particularly poor (Exhibit 4.2). 


4.48 We found that reviews completed in 2010 that did not meet 
service standards took significantly longer than the standard to 
complete. The following are examples: 


* Reviews of abbreviated new drug submissions (generic drugs) that 
did not meet service standards took, on average, 353 days 
(173 days longer than the 180-day standard). 


* Reviews of DIN-A submissions (primarily over-the-counter drugs) 
that did not meet service standards took, on average, 539 days 


(329 days longer than the 210-day standard). 


* Reviews of post-market manufacturing and safety-related labelling 
change submissions that did not meet service standards took, on 
average, 251 and 158 days, respectively (161 and 68 days, 
respectively, longer than the 90-day standard). 
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Exhibit 4.2 Health Canada is not meeting its own service standards for reviewing most drug submissions 


Submission type 


Description 


Service standard for -—— 
2009 


first review decision* 


- Percentage that meet 
service standards 
(target 90%) 


mt 


New drugs Se . 
New drug Required for new drugs that have not been sold in Canada | 300 calendar days 70% 70% 
submission for a sufficient time and in sufficient quantity to establish 

their safety and effectiveness—includes clinical trial 

information and details on production, packaging, 

labelling, conditions for use, and side effects 
Supplemental new Required if 300 calendar days 92% 91% 


drug submission 


¢ substantial changes are made to a drug previously 
approved as a new drug submission, including dosage 
form, drug strength, method of manufacture, and 
labelling; or 


e the manufacturer wishes to expand the diseases or 
conditions the product is approved to treat 


Generic drugs 


a ee 


180 calendar days 


Abbreviated new Required for new generic drugs; submissions must include 36% 12% 
drug submission evidence that the generic is equivalent to the existing 

patented drug (delivers the same amount of medicinal 

ingredient and at the same rate) 
Supplemental Required if changes are made to a new generic product 180 calendar days 71% 5% 


abbreviated new 
drug submission 


that was previously approved as an abbreviated new drug 
submission, including method of manufacture, labelling, 
recommended route of administration, or a new indication 


Over-the-counter drugs 


a 


ea ee ee ee ee ee ee 


Drug identification 
number (DIN) 
applications 


counter drugs that comply with existing drug labelling and 
that do not require additional supporting scientific data. 


A DIN-A application is used most commonly for over-the- 210 calendar days 19% 36% 
counter drugs that have established safety records but that 

require additional supporting data and a clinical review. 

A DIN-F application is used most commonly for over-the- 45 calendar days 93% 83% 


ENS RS es Pe ee ee 


Post-market (notifiable) changes 
Chemistry or Manufacturers must inform Health Canada of changes toa | 90 calendar days 9% 14% 
manufacturing drug that could adversely affect its safety, purity, potency, 
changes or effectiveness. 

se 
Labelling and Manufacturers must inform Health Canada of changes toa | 90 calendar days 57% 70% 


product monograph 
changes 


drug that could adversely affect its safety, purity, potency, 
or effectiveness. 


Note: These service standards exclude the screening times to assess the completeness of a submission. 
t in a request for additional information from the manufacturer, a rejection of the submission, or an approval. 


* A first decision can resu 
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Risk communication—The development and 
dissemination of information about potential or 
existing health risks, to enable patients and their 
health care professionals to make better 
informed decisions about their health. 


Health Canada’s four review bureaus—Drug 
review responsibilities are divided among four 
review bureaus based on the type of review 
required, the drug’s therapeutic class, and 
whether the drug will be available by prescription 
or over the counter. 


Three clinical review bureaus are responsible for 
assessing drugs that treat specific diseases, 
such as cancer or HIV. The fourth bureau reviews 
submissions for generic drugs and reviews the 
manufacturing and chemistry component of all 
drug submissions. The four review bureaus 
comprise eleven more specialized review units. 
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4.49 According to Health Canada, a number of factors can contribute 
to poor review performance. For example, drug reviewers are often 
required to perform duties other than reviewing submissions, such as 


* assessing the potential health risks posed by defective or illegal 
products; or 


* contributing to related risk communications, so the public is 
informed of potential safety risks. 


4.50 We found that Health Canada has taken a number of steps to 
improve its review performance. Foremost among these was a new cost 
recovery framework, implemented in April 2011. The Department 
expects that the additional revenue, obtained by charging the 
pharmaceutical industry increased fees, will improve review times. The 
new user fees will be governed by the User Fees Act, which includes 
penalties (of up to 50 percent of the fees) for the Department if it does 
not meet its service standards. This means that the Department would 
receive less revenue from user fees, which could in turn reduce its 
resources for reviewing submissions. 


4.51 We also found that the Department launched an initiative 

in June 2010 to develop a consistent approach to the use of foreign 
regulatory information in order to improve review efficiency. Health 
Canada’s four review bureaus have each explored different approaches 
to reduce existing backlogs and to improve review performance, such as 
assessing workloads to determine the actual number of hours required 
to review drug submissions and the time spent on non-review activities. 
They expect this work will help streamline review processes and 
improve planning. 


4.52 Canadians and health care professionals benefit from timely 
access to safe and effective drugs. It is important that Health Canada 
take the necessary time to properly evaluate a drug for safety and 
efficacy, but delays in approving new drug submissions mean that 
access to the potential benefits of these drugs is delayed. The untimely 
review of abbreviated (generic) new drug submissions may limit access 
to more affordable treatments. The delayed review of post-market 
change submissions, particularly those for labelling changes to address 
potential safety issues identified by the pharmaceutical industry or by 
the Department, means that new safety information may not be 
provided to Canadians as quickly as possible. 


453 Recommendation. Health Canada should ensure that it meets 
service standards for the review of all drug submission types—by giving 


due consideration to the appropriate allocation of additional resources 
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from increased fees charged to industry, to the use of foreign regulatory 
information, and to streamlining its review processes. 


The Department’s response. Agreed. Revenues from recently 
updated user fees will allow the Department to meet its 
well-established and internationally recognized performance standards. 
The Department will closely monitor its performance. It will continue 
to seek process efficiencies, such as increasing the leverage of external 
scientific expertise and the use of foreign regulatory information. The 
Department will begin piloting an approach to enhance and formalize 
the use of foreign reviews through standard operating procedures and 
guidance for industry in the fall of 2011 and will complete an 
evaluation of the pilot by 31 March 2014. 


Health Canada has not assessed whether its review bureaus interpret and apply 
review procedures and guidelines consistently 


4.54 We examined whether Health Canada had established a quality 
assurance system for its review of drug submissions. In a strategic plan 
for these reviews—Therapeutic Products Directorate Strategic Plan, 
2006-2009, The Way Forward—the Department acknowledged the 
importance of consistent, timely, and high-quality review decisions. 


4.55 We found that the Department has the following key 
components of a quality assurance system: 


* standard operating procedures, 
* guidelines for drug reviewers, 

* review templates, 

* training programs, and 


* management review of individual files. 


4.56 However, we also found that Health Canada has not assessed 
whether review procedures, guidelines, and templates were 
consistently interpreted and applied across the four different review 
bureaus responsible for conducting reviews of drug submissions. Such 
an assessment would allow the Department to identify inconsistencies 
between the different review bureaus that may affect the timeliness 
and quality of review decisions. 


4.51 Recommendation. Health Canada should regularly assess 
whether the procedures and guidelines, which were established to 
ensure timely, consistent, and high-quality review decisions, are 
interpreted and applied consistently by all four review bureaus. 
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a drug. 
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The Department’s response. Agreed. The Department will develop a 
system to regularly assess and ensure the use of procedures established 
to ensure timely, consistent, and high-quality review decisions 

by 31 December 2012. Implementation of the system, which will 
include assessment of compliance with procedures and consistency 

of interpretation across organization review units, and necessary 
corrective mechanisms to ensure consistent use, will be completed 


by 31 December 2013. 


More information on Health Canada’s reviews of drug submissions needs to be 
made available to Canadians 


4.58 We examined several of Health Canada’s commitments to 
increase the transparency of review decisions and the amount of 
information it makes available to Canadians about approved drugs. 
These commitments are consistent with government-wide directives to 
foster understandable and responsive regulation through inclusiveness, 
transparency, accountability, and public scrutiny. It is important that 
Canadians have access to information so they are able to make 
informed decisions about the drugs they use. 


4.59 ‘To determine whether Health Canada had fulfilled its 
commitments for increasing the transparency of review decisions, we 
examined the availability and the timeliness of the public disclosure of 
key documents to its website (product monograph, notice of decision, 
summary basis of decision) for all 34 new active substances approved 
in 2009 and 2010. We found that, while it had met its timelines for 
posting product monographs, the Department had not consistently 
met its timelines for posting notice of decision or summary basis of 
decision documents (Exhibit 4.3). 


4.60 Health Canada occasionally attaches conditions to a drug’s 
approval, asking the manufacturer to carry out additional post-market 
studies. We found that the Department does not disclose the timelines 
established for fulfilling these conditions, nor does it report on the 
progress made by manufacturers in fulfilling these conditions. We 
noted that the US Food and Drug Administration does provide 
updates on the status of post-market conditions. 


4.61 Health Canada told us that, until recently, manufacturers were 
not asked to report regularly on their progress in fulfilling post-market 
conditions. Since June 2011, manufacturers have been asked to provide 


annual updates, but they are under no regulatory obligation to do so. 


4.62 Health Canada is also not disclosing information on drugs that it 


rejects or on drugs that the manufacturer withdraws from the review 
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process. Health care providers have the discretion to prescribe a drug for 
conditions that the drug has not been authorized to treat. Therefore, it is 
important that they be informed when the Department rejects a 
marketed drug for a new use, so they understand the Department’s 
concerns. We also noted that the European Medicines Agency, which is 
responsible for the scientific evaluation of medicines for use in the 
European Union, discloses information related to rejections and 
withdrawals of drug submissions. 


Exhibit 4.3 The Department did not consistently meet its own timelines for posting review documents for drugs with new active substances 


5 Rea She ROS BS oe 2 oe Sixsascd Sas } 
Product Describes the health claims, indications, At the time 28* ¢ 28 met the timeline 
monograph and conditions for the safe and effective use of market 

of a drug. It also includes other important notification 
information, such as safety warnings, 
precautions, adverse reactions, and 
interactions with other drugs. 
Notice of Outlines in a one-page summary the Within 6 weeks of See ¢ 20 met the timeline 
decision authorization received and general drug approval 
information related to the approved drug. 2 1A took att average of 
3 weeks longer 
SE 
Summary basis | Outlines the scientific and benefit or risk Within 20 weeks Site ¢ 15 met the timeline 
of decision considerations that factor into Health of drug approval 
Canada’s decision to approve a drug. = 15 t00K ap avetage of 
12 weeks longer 
¢ 1 was not yet posted 
eas eS beseash 


* 


Five of the new drugs approved during the period subject to audit have not been marketed in Canada, and one of the new drugs was discontinued. Therefore, 
a product monograph was not required for these six drugs. 


** Three new drugs approved in the period subject to audit were not considered to be new active substances. Therefore, they were not eligible for a Notice of 
Decision or a Summary Basis of Decision document. 


4.63 Recommendation. Health Canada should disclose information 
related to new drug approvals in a timely manner and improve the 
transparency of “approvals with conditions,” rejections, and withdrawals 
of new drugs so that Canadians and health care professionals can access 
information about these drugs. 


The Department’s response. Agreed. The Department will improve the 
transparency of approvals with conditions, rejections, and withdrawals to 
the Canadian public. The Department will consult with stakeholders in 
fall 2011 about expanding its public communications on post-approval 
decisions for marketed health products to include information on 
approvals with conditions, rejections, and withdrawals, with a view to 
disclosing additional information by June 2012. 
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Health Canada has not determined what measures are necessary to manage 
conflict-of-interest risks for drug reviewers 


4.64 In the federal government, “conflict of interest” refers to a 
conflict between the public service duties and private interests of 
public servants. In the 2010 Fall Report of the Auditor General, 
Chapter 4, Managing Conflict of Interest, we noted that 


... conflicts of interest bring into question the integrity and 
fairness of decisions made by public servants. If not properly 
addressed, conflicts of interest can increase the level of distrust 
and cynicism toward government and, over time, impact the 
legitimacy and effectiveness of government actions. 


4.65 The government’s Values and Ethics Code for the Public Service 
requires that departments establish measures to manage conflicts of 
interest. To determine whether Health Canada had systems to manage 
conflict-of-interest risks to the drug submission review process, we 
examined its Code of Conduct and its conflict-of-interest guidelines, 
and interviewed key entity officials. The audit was not designed to find 
cases of officials being in a conflict of interest, and we did not find any 
such cases. 


4.66 We found that Health Canada’s code of conduct and conflict-of- 
interest guidelines are consistent with the government’s Values and 
Ethics Code for the Public Service. However, in 2010, the Department 
had not complied with the code’s requirement to issue an annual 
reminder to employees of their conflict-of-interest obligations and, at 
the time of our audit, the Department had not issued this reminder 


for 2014, 


4.67 Compliance measures beyond those specified in the Values and 
Ethics Code for the Public Service are permitted if a department 
believes they are necessary to address its specific responsibilities. 
Officials responsible for reviewing drug submissions routinely handle 
commercially sensitive information that could be used for personal 
gain, and they are directly involved in making decisions that could 
have significant commercial benefits. However, we found that the 
Department has not determined what measures are necessary to 


address these risks. 


4.68 In our 2010 Fall Report, Chapter 4, Managing Conflict of 
Interest, we noted that some government departments had developed 
conflict-of-interest requirements for specific work assignments. For 
example, Natural Resources Canada requires staff assigned to the 


management of contribution agreements to acknowledge in writing 
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that they understand what to do to avoid and disclose conflicts of 
interest for each agreement in which they are involved. 


4.69 We also noted that the European Medicines Agency requires 
that its employees’ declarations of interests be updated annually and 
that its drug reviewers’ declarations be checked each time they are 
assigned to review a new drug submission. 


4.10 Recommendation. Health Canada should assess the risks 
posed by conflicts of interest to the drug review process, determine 
what measures are necessary to manage these risks, and implement 
those measures. 


The Department’s response. Agreed. The Department requires 
disclosure of potential conflicts of interest at the initiation of 
employment, and employees are subject to the Values and Ethics Code 
for the Public Service. The Department will determine if there are any 
particular risks posed by potential conflicts of interest in the drug 
review process by 31 March 2012 and, if necessary, develop and 
implement additional measures by 30 September 2012. 


Monitoring post-market safety 4.71 Health Canada monitors the safety of marketed drugs by 
collecting, analyzing, and assessing domestic adverse drug reaction 
reports that are submitted by the pharmaceutical industry, health 
professionals, and consumers. The Department also collects foreign 
adverse drug reaction reports that are submitted by the pharmaceutical 
industry. In addition, it reviews scientific literature, as well as actions 
taken by other regulators, to address safety concerns identified from 
these sources. The Department uses this information to determine 
whether further action is needed to protect the public, including 


* conducting full assessments of potential safety issues, 


* recommending that manufacturers revise product labels and 
working with manufacturers to implement these revisions, and 


* communicating new safety information to health care providers 
and the public. 


4.12 We examined Health Canada’s approach to monitoring the 
safety of drugs marketed in Canada. According to its post-market 
surveillance strategy, the Department is responsible for ensuring that 
the benefits of using a drug outweigh the risks. It fulfills this 
responsibility by gathering and assessing safety information, from a 
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variety of sources, and by implementing measures necessary to reduce 
the safety risks associated with marketed drugs. To determine whether 
the Department uses a risk-based approach for monitoring marketed 
drugs, we examined its processes for obtaining, assessing, and acting on 
drug safety information. 


Health Canada has recently taken additional steps to actively monitor drug safety 


473 In 2010, Health Canada received about 30,000 domestic adverse 
drug reaction reports and about 330,000 foreign reports. The vast 
majority of these reports were submitted by the pharmaceutical 
industry, which is required to report adverse reactions to the 
Department. Adverse drug reaction reports are an important 
component of monitoring drug safety. 


474 We found that Health Canada does not have mechanisms to 
receive these adverse reaction reports electronically, and it has not 
entered the foreign reports it receives each year in its post-market 
adverse drug reaction database. Adverse drug reactions captured in 
this database may help identify potential safety issues. Although the 
Department can consult foreign adverse drug reaction reports when 
necessary, it does not regularly analyze these reports or search them 
electronically to detect emerging safety issues. Canada’s small 
population reduces the likelihood of serious, rare adverse drug 
reactions being identified in this country; therefore, the capacity to 
search and analyze foreign reports electronically would contribute to 
more comprehensive safety monitoring. 


4.715 The Department expects to begin electronic reporting pilots in the 
2011-12 fiscal year. When electronic reporting is fully implemented, the 
Department believes it will have the capacity to electronically capture 
and analyze the foreign reports it receives each year. 


4.76 We found that Health Canada had recently implemented 
strategies to electronically search its domestic adverse drug reaction 
reports, to better detect adverse reactions for specific drugs, and to 
systematically monitor those adverse reactions that are rare, serious, 
and often linked to marketed drugs. A similar approach to monitoring 
adverse drug reaction data for vulnerable populations (for example, 


children) has yet to be implemented. 


4.77 Due to the limitations of adverse drug reaction reports, the 
World Health Organization has encouraged regulators to incorporate 


active surveillance into their monitoring activities. We found that, 
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Risk management plan—A set of monitoring 
activities and interventions by the manufacturer 
designed to identify, characterize, prevent, or 
minimize risks relating to drugs and designed to 
assess the effectiveness of those interventions. 


since 2009, Health Canada has taken several steps to increase active 
monitoring, including the following: 


* establishing formal working groups to systematically consider 
potential safety issues that were identified by other regulators, 
reported in scientific journals, or identified by industry; 


* implementing an inspection program to ensure that companies 
comply with regulatory requirements to report adverse drug 
reactions; and 


* reviewing risk management plans voluntarily submitted by the 
pharmaceutical industry, as part of some drug submissions, which 
may identify potential safety issues and include manufacturers’ 
commitments to actively monitor the drug. 


478 The Department has acknowledged that it had not yet set up 
monitoring systems to identify adverse drug reactions in patients who 
are taking certain drugs or to detect adverse reactions in health care 
settings, such as emergency departments, where those suffering an 
adverse reaction may seek treatment. However, the Department has 
recently launched an initiative to identify more active surveillance 
activities and to determine their potential value. 


Health Canada’s assessment of, and response to, potential safety issues 
is not timely 


479 Assessing safety issues. Health Canada uses information 
obtained through its monitoring activities to assess whether there may 
be a safety issue with a specific drug. Depending on the results of its 
assessment, it may recommend changes to the drug’s label, issue a risk 
communication to the public or health care professionals about the 
risks of using the drug, or do both. 


4.80 In certain instances, Health Canada may also withdraw a drug’s 
market approval. It is important to note that the Department has 
indicated that after it has approved a drug for sale, it has limited 
authority to require that manufacturers update drug labels or issue risk 
communications about new safety concerns. Overall, we found that 
the Department did not assess potential safety issues, detected through 
its monitoring activities, in a timely manner. Therefore, changes to 
drug labels and risk communications were also not timely. 


4.81 In 2009 and 2010, Health Canada completed 99 assessments of 
potential safety issues, which were identified through its post-market 
monitoring activities. The Department had established draft risk-based 
timelines for these assessments. Of the 99 assessments, 54 resulted in 
recommendations about updating the labelling for specific drugs or 
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classes of drugs and issuing risk communications. Almost half of the 
assessments that resulted in recommendations were triggered by 
actions taken by other regulators (Exhibit 4.4). 


Exhibit 4.4 The greatest number of Health Canada safety recommendations resulted from actions 
by foreign regulators 


Actions of foreign regulators 25 
Safety information from scientific literature 15 
Adverse drug reaction reports, and previous Health S| 
Canada assessments of potential safety issues 

Safety information provided by manufacturers iy: S} 
Total 54 


4.82 We reviewed all 54 of these assessments (Exhibit 4.5). It is 
important that these assessments be completed in a timely manner, so 
if there are safety risks they can be confirmed and communicated to 
Canadians. 


Exhibit 4.5 Health Canada does not consistently meet its targets for completing safety assessments 


High—Potential safety issues 
that, if confirmed, will likely 
require an intervention. These 
include adverse drug reactions 
that were unknown, 
unlabelled, or insufficiently 
labelled. 


80 working 
days 


Medium—Potential safety 
issues that are not as serious 
or not as unexpected, but still 
require a comprehensive 
assessment. If confirmed, 
these safety issues are likely 
to require labelling changes. 


130 working 


days ae 


Low—Potential safety issues 
that are well known or 

labelled. These issues are not 
likely to affect the way a drug 
is used. 


200 working 
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4.83 Of the 54 assessments we examined, 34 were completed within 
the established timelines. However, the Department’s approach to 
measuring its performance does not consider the following: 


* amount of time a potential safety issue may wait before an 
assessment begins; 


* amount of time an assessment may be placed on hold; 


* amount of time needed to obtain additional information from 
external parties (for example, manufacturers or other regulators); 
and 


¢ total number of calendar days, instead of working days, taken to 
complete the assessment. 


4.84 When these factors are considered, Health Canada took at least 
one year to complete 34 of its 54 assessments. In some cases, it took 
significantly longer. For example, 5 medium-priority assessments 
required more than two years to complete, and 1 of the 5 required 
more than three years to complete. 


4.85 Although Health Canada did not conduct any high-priority 
safety assessments in 2009 and 2010, we noted that two drugs were 
voluntarily withdrawn by manufacturers from the Canadian market 

for safety reasons during that period. Both withdrawals occurred after 
new studies confirmed that the risks of using the drugs outweighed any 
benefits. One of these drugs was withdrawn by the manufacturer before 
the Department had completed an ongoing safety assessment, and 

the other was withdrawn without an assessment being conducted. 

In both cases, the Department publicly announced the withdrawals at 
approximately the same time as the US Food and Drug Administration. 


4.86 Updating drug labels and communicating health risks. 
According to its post-market surveillance strategy, Health Canada 

is responsible for managing identified safety risks, which includes 
communicating health risks to health care professionals and the public, 
and working with manufacturers to update drug labels with the most 
recent safety information. Of the 54 safety assessments we examined, 
Health Canada had recommended 51 labelling updates and 24 risk 
communications. It is important that these recommendations are 
implemented so that health care providers and consumers have access 
to the most up-to-date safety information available. 


4.87 We found that there is no systematic process to implement 
recommendations for labelling updates. This is important because 
officials responsible for making safety-related recommendations are not 
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the same officials responsible for working with the drug manufacturers 
to implement these recommendations. We found that the officials 
responsible for implementing recommendations often do not document 
whether they agree or disagree with recommended labelling updates or 
how they intend to implement the recommendations, including what 
the proposed timelines are for implementation. 


4.88 Once the Department has decided to implement a 
recommendation to update a drug’s label to include new safety 
information, its officials are responsible for starting the process, by 
notifying the manufacturer of the requested update. We found that 
the Department has not established timelines for issuing these 
notifications. 


4.89 Health Canada needed to notify manufacturers about 38 of 
the 51 recommended label updates. It did not need to notify 
manufacturers about the other 13 because in some of those cases, the 
manufacturer had already updated the labelling, and, in others, the 
Department was already reviewing a manufacturer’s submission to 
address the recommendation. 


4.90 We found that for 12 of these 38 recommended updates, it took 
Health Canada between 3 and almost 20 months to issue notifications 
to manufacturers. We also found that the Department had not yet 
notified the manufacturer of another 6 recommended label updates, 
even though between 6 and 28 months had passed since the 
recommendations were first made. Officials told us that the 
Department may delay notifying manufacturers of requested label 
updates if it does not have the resources available to review all the 
submissions that will result from the request. For example, according to 
the Department, one label update that affects a class of drugs could 
generate well over 100 notifiable change submissions. 


4.91 We also found that labelling recommendations are not directly 
communicated to the departmental officials who are responsible for 
working with the pharmaceutical industry to update labels for generic 
drugs. As a result, these officials may not be informed of key safety 
issues, and labels on generic drugs may not be updated in a timely 
manner. According to IMS Brogan, 57 percent of prescriptions 
dispensed by Canadian retail pharmacies in 2010 were for generic 
drugs. The case study on a label change for a drug to treat epilepsy 
shows the impact of not formally tracking safety recommendations— 
that is, how undefined processes and poor communications can result 


in inconsistent labelling between brand name and generic drugs. 
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Case study—Label change for a drug to treat epilepsy 


In August 2009, Health Canada completed a safety assessment on a drug approved to 
treat epilepsy and migraine headaches, which was also used off-label to treat several 
psychiatric conditions and to promote weight loss. The completed assessment 
recommended that the drug’s label be updated to reflect newly identified risks related 
to increased incidence of birth defects. 


In September 2009, the Department sent a letter to the brand-name manufacturer of 
the drug requesting that the drug’s label be updated to reflect this new safety 
information. In April 2010, the label for the brand name drug was revised. 


According to Health Canada, generic drug manufacturers should update safety 
information so generic drug labels are consistent with those for brand-name drugs. The 
Department has a practice of alerting generic manufacturers when a brand-name drug - 
revises Its labelling to address new safety issues. 


Health Canada officials responsible for working with the generic drug industry were 
not notified of the required labelling changes until February 201 1—almost one year 
after the brand name drug changed its product monograph. Hence, the Department 
did not inform the 12 generic manufacturers with outdated product monographs of the 
need to update these documents until March 2011. The Department did not need to 
inform two other manufacturers of the drug, because they had already updated their 
product monographs. As of 31 May 2011, they remain the only generic drug 
manufacturers marketing this drug in Canada with updated product monographs. 


4.92 Safety assessments may also recommend that risk 
communications be issued to inform the public of new drug safety 


‘information. We found that of the 24 recommended risk 


communications that we examined, 11 were not issued by Health 
Canada until six months after the assessment had been completed. In 
three cases, we found that the Department took more than a year to 
issue a recommended risk communication. 


4.93 According to Health Canada, in less urgent cases, the 
Department does not issue a risk communication until a drug’s label 
has been updated to reflect the most recent safety information. 


4.94 As noted earlier, the Department does not complete its safety 
assessments, or notify manufacturers of requested label updates, in a 
timely manner, and it does not consistently meet its service standards 
for the review of notifiable change submissions (Exhibit 4.2). As a 
result, a significant amount of time can pass before recommended risk 
communications are issued to Canadians. 


4.95 Overall, we found that for 11 of the 24 recommended risk 
communications that we examined, it took the Department more than 
two years to assess the potential safety issue, update the drug’s label 
(where necessary), and issue the risk communication. 
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Case study—Delayed risk communication for two commonly used drugs 


In 2006, the US Food and Drug Administration released a public risk communication 
stating that the therapeutic effectiveness of daily low-dose acetylsalicylic acid (ASA) 
therapy was potentially reduced when ibuprofen was taken at the same time. Health 
Canada started to assess this potential safety issue in January 2006 and finished in 
September 2009. It recommended that labels for products containing acetylsalicylic 
acid and ibuprofen be updated and that a risk communication be issued to inform 
Canadians of this new safety information. As of May 2011, not all labels for the 
several hundred affected products had been updated, and Health Canada had not 
issued the recommended risk communication. 


4.96 Recommendation. Health Canada should improve the 
timeliness of safety assessments and the implementation of related 
recommendations to update labels and to issue risk communications, 
so Canadians and health care professionals can be informed of new 
drug safety information in a timely manner. 


The Department’s response. Agreed. The Department will improve 
the timeliness of safety assessments by fully implementing and 
respecting related performance standards by December 2013. This 
involves reviewing baselines for completing safety reviews. 


497 Recommendation. Health Canada should establish a systematic 
process to manage safety assessment recommendations for marketed 
drugs, to ensure that recommendations are dealt with appropriately 
and in a timely manner. 


The Department’s response. Agreed. The Department will establish a 
systematic process to implement safety assessment recommendations 
for marketed drugs. The Department has already begun developing a 
standard operating procedure in this regard. The standard operating 
procedure will include new formalized tracking systems (currently 
under development) and necessary implementation procedures 
following a safety assessment recommendation. This will be in place 


by 31 March 2013. 


4.98 Assessing the effectiveness of risk communications. In addition 
to communicating safety risks to Canadians in a timely manner, it is 
important for Health Canada to know whether its risk communications 
are effective and have an impact on targeted audiences. In Planning for 
Our Future: Federal Regulatory Post-Market Surveillance Strategy 
2007-2012, the Department includes a commitment to evaluate the 


effectiveness of its risk management actions. 
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4.99 We found that Health Canada has not assessed the effectiveness 
of risk communications but has begun to examine this issue. For 
example, the Department commissioned a study of how health care 
professionals use its risk communications: specifically, the impact of 
these communications on prescribing patterns in Canada. Officials 
told us that assessing the impact of risk communications on behaviour 
is also challenging for foreign regulators, because there is no widely 
accepted approach to measuring the effectiveness of risk 
communications. 


Enforcing compliance with 4.100 Health Canada’s compliance and enforcement activities are 
the regulations © conducted by inspectors in its regional offices. The Department 
maintains a national inspection program that examines the operations 
of various drug establishments, such as manufacturers, wholesalers, 
and distributors—to verify that the pharmaceutical industry is 
complying with the internationally accepted Good Manufacturing 
Practices, which are set out in the Food and Drug Regulations. 


4.101 Serious deficiencies identified during an inspection can result in 
a request for a product recall or the suspension of an establishment 
licence. Health Canada also follows up on reported complaints or 
concerns, such as those related to the illegal sale of pharmaceutical 
drugs or the quality or safety of those available for sale. The Department 
also monitors the industry’s implementation of product recalls. 


Actions have been taken to make inspections of drug establishments 
more risk-based 


4.102 Health Canada inspects Canadian drug establishments to verify 
that they comply with the good manufacturing practices required by 
the Food and Drug Regulations, which reduces the likelihood that 
Canadians will be exposed to substandard drugs. The Department’s draft 
Compliance and Enforcement Risk Evaluation Guide: An Approach to 
Decision Making specifies that compliance and enforcement activities 
associated with regulated products or activities need to be appropriate 
and proportional to the risks. 


4.103 We found that the Department has established risk-based 
inspection cycles for drug establishments based on the nature of their 
activities. For example, drug manufacturers were inspected every two 
years, whereas lower-risk establishments, such as distributors, were 
inspected every three years. 


4.104 In June 2010, Health Canada completed a review of its drug 
establishment inspection program and recommended that the program 
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be more risk-based. As a first step, in April 2011, the Department 
eliminated renewal inspections that are normally conducted when 
annual establishment licences expire. It expected that this change 
would enable inspectors to focus on high-risk establishments, as 
opposed to those requiring an inspection as part of the licence renewal 
process. Other recommendations arising from this review were also 
being implemented. 


4.105 Foreign manufacturers. According to Health Canada, 
approximately 80 percent of health products used by Canadians are 
manufactured in other countries. The Department established 
mutual recognition agreements with 26 international partners to 
ensure that drugs manufactured in these jurisdictions and imported 
into Canada comply with the Good Manufacturing Practices required 
by the Food and Drug Regulations. 


Complaints about marketed drugs are not prioritized consistently 


4.106 We examined whether Health Canada used a risk-based 
approach to follow up on complaints about drugs, and whether 

it verified compliance with the Food and Drug Regulations. The 
Department’s draft Compliance and Enforcement Risk Evaluation 
Guide: An Approach to Decision Making specifies that compliance 
and enforcement activities associated with a regulated product or 
activity be appropriate and proportional to the risk. Verifying the 
regulatory compliance of marketed drugs is critical to ensuring that 
Canadians are not exposed to unsafe drugs. 


4.107 ‘To determine whether Health Canada used a risk-based 
approach, we 


* reviewed its performance reports and standard operating 
procedures, for its compliance verification activities; 


* interviewed its key officials in Ottawa and in its three largest 
regional operational centres; and 


* reviewed a representative sample of 50 files related to drug- 


specific complaints that were also closed in 2009 or 2010. 


4.108 In 2009 and 2010, the Department received almost 

800 complaints concerning specific drugs from consumers, health care 
professionals, the pharmaceutical industry, and other stakeholders. 
Health Canada had established risk-based standard operating 
procedures for prioritizing reported cases of non-compliance to ensure 
that compliance and enforcement actions are initiated within specified 
timelines. However, we found that these procedures were not 
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implemented consistently. In 27 of the 50 complaint files we reviewed, 
we found that the Department had not prioritized these complaints 
using its standard operating procedures. Therefore, the Department 
could not demonstrate that compliance and enforcement actions were 
initiated in a timely manner. For 12 of the 23 complaints that were 
prioritized in accordance with procedures, we found that the 
Department had initiated compliance and enforcement actions 
within the established timelines. 


4.109 Recommendation. Health Canada should consistently apply its 
risk-based standard operating procedures, so the priority of the drug 
complaints it receives is properly documented and addressed in a 
timely manner. 


The Department’s response. Agreed. The Department strives to 
ensure that complaints received are addressed in a timely manner and 
are properly documented. The Department will review its processes to 
ensure better documentation, conduct training sessions, and 
implement more robust performance monitoring by 31 March 2013. 


Conclusion 


4.110 We conclude that Health Canada has fulfilled its responsibilities 
for the timely review of clinical trial applications and amendments. 
However, it has not adequately fulfilled its key responsibilities for 
verifying the regulatory compliance of authorized trials. Although 
Health Canada has a risk-based clinical trial inspection strategy, it does 
not have all the information it needs to make comparative risk-based 
decisions about which sites to inspect, and it does not regularly report 
on its clinical trial inspection activities. The Department had also not 
met its clinical trial inspection target in 2009 or 2010. Furthermore, 
the Department does not document its approach to monitoring 
adverse drug reactions in clinical trials to ensure that it focuses on the 
trials that posed the greatest risk. It did not fulfill long-standing 
commitments to increase the transparency of authorized clinical trials. 


4.111 Health Canada did not review most drug submissions within its 
established service standards and did not assess whether its four review 
bureaus consistently interpret and apply procedures designed to support 
timely, consistent, and high-quality reviews of drug submissions. The 
Department increased the amount of information publicly available on 
approved drugs and on its rationale for approving these drugs. However, 
this information could be timelier. The Department does not disclose 
information on drug submissions that were rejected or withdrawn. 
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4.112 Health Canada’s conflict-of-interest guidelines are consistent 
with the government's Values and Ethics Code for the Public Service. 
However, in 2010, the Department had not complied with the code’s 
requirement to issue an annual reminder to employees of their 
conflict-of-interest obligations. At the time of our audit, the 
Department had not yet issued this reminder for 2011, nor had it 
determined the necessary measures to address the conflict-of-interest 
risks specific to its review activities. Some federal government 
departments and a major regulator of pharmaceuticals have developed 
additional measures to manage conflicts of interest for specific activities. 


4.113 Health Canada has not adequately fulfilled its key responsibilities 
for monitoring the safety of marketed drugs. It recently took steps to 
actively monitor drug safety, and it is developing the capacity to 
process and analyze foreign adverse drug reaction reports that it 
receives from the pharmaceutical industry. The Department developed 
a risk-based approach to assessing the safety issues it identified through 
its monitoring activities, and it has established timelines for this work. 
However, it does not complete these assessments or communicate 
safety concerns in a timely manner, and it has not established 
systematic processes for implementing the recommendations that 
result from these assessments. 


4.114 Health Canada has fulfilled its key responsibilities for verifying 
the regulatory compliance of industry. It takes a risk-based approach to 
inspecting drug establishments and is working to strengthen this 
approach. However, although the Department developed a risk-based 
approach for following up on complaints about specific drugs, it does 
not apply it consistently. 


4.115 In summary, we examined key Health Canada responsibilities 
involving timeliness, consistency, transparency, conflict of interest, and 
risk-based post-market activities. We found that the Department has 
not adequately fulfilled most of these key responsibilities related to 
clinical trials, submission reviews, and post-market activities for 
pharmaceutical drugs. 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these 
standards as the minimum requirement for our audits, we also draw upon the standards and practices of 
other disciplines. 


Objectives 


The overall audit objective was to determine whether Health Canada fulfilled its key responsibilities 
related to clinical trials, submission reviews, and post-market activities for pharmaceutical drugs. 


The audit sub-objectives were to determine whether the Department fulfilled its key responsibilities for 


- conducting timely clinical trial application reviews; 


supporting the transparency of authorized clinical trials, and verifying their regulatory compliance; 


conducting timely and consistent submission reviews; 


supporting the transparency of review outcomes; 


implementing systems to manage conflicts of interest; 


monitoring the safety of marketed pharmaceuticals; 


communicating safety concerns in a timely manner; and 


verifying the regulatory compliance of industry. 


Scope and approach 


Our audit focused on the three Health Canada directorates that are involved in the regulation of 
pharmaceuticals. We examined the timeliness of reviews for clinical trial applications and for drug 
submissions, received from the pharmaceutical industry, for market authorization and post-market 
changes. Where necessary, we conducted gap analyses to assess the completeness of the data used by the 
Department to measure its review performance. We also verified the accuracy of the Department’s data by 
conducting a file review of a representative sample of clinical trial applications and drug submissions. 


We also examined whether the Department had established the necessary quality assurance systems to 
support Consistency among its review activities and what efforts it had made to ensure the transparency of 
clinical trials by disclosing information about those that were in progress, abandoned, or completed. We 
also reviewed various departmental initiatives to increase the transparency of its drug submission reviews. 


We examined the Department’s safety monitoring activities for drugs being tested in clinical trials and for 
those already marketed, to determine whether the Department completed its assessments and ensured 


that safety-related recommendations, such as labelling updates and risk communications, were 
implemented in a timely manner. 
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We also examined compliance and enforcement activities to determine whether the Department had 
employed a risk-based approach to its inspection of clinical trial sites and pharmaceutical establishments 
and to its follow-up of product-specific complaints or concerns it had received from various stakeholders. 
As part of this work, we examined a representative sample of complaints. The sample size was sufficient to 
conclude on the sampled population with a margin of error of +10 percent 18 times out of 20. 


We interviewed key Health Canada officials involved with regulation of pharmaceuticals at headquarters 
and in the larger regional offices and met with representatives from the pharmaceutical industry and the 
health care field. We reviewed documentation, including regulations, strategic and operational plans, 
program reviews, and reports filed by the Department’s inspectors in the field. 


We did not examine the following: 


regulation of biologics (products derived from living sources, such as blood and vaccines), 
radiopharmaceuticals (radioactive products used to diagnose illness), or disinfectants; 


completed reviews of drug submissions to determine whether drugs approved by the Department were 
safe or reviewed in a consistent manner; 


regulation of direct-to-consumer advertising or efforts at the border to prevent the importation of 
prescription or counterfeit pharmaceutical products; and 


the Special Access Program or Patented Medicines Prices Review Board. 


Criteria 


Health Canada reviews clinical trial applications in a Guidance for Industry: Management of Drug Submissions, 


timely fashion. Health Canada 
Food and Drug Regulations 


The Department publicly discloses information related to clinical Blueprint for Renewal Il: Modernizing Canada’s Regulatory 
trials that are in progress, abandoned, or completed. System for Health Products and Food, Health Canada 


Health Products and Food Branch Strategic Plan, Health 
Canada, 2007-2012 


Opening the Medicine Cabinet: First Report on Health Aspects 
of Prescription Drugs by the House of Commons Standing 
Committee on Health 
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The Department uses a risk-based approach to inspect clinical ¢ Framework for the Management of Risk, Treasury Board 


trial sites and to monitor adverse drug reactions in order to verify | , Decision-Making Framework for Identifying, Assessing and 
compliance with the Food and Drug Regulations. Managing Health Risks, Health Canada 


Health Product and Food Branch’s Compliance and 
Enforcement Risk Evaluation Guide: An Approach to Decision- 
Making, Health Canada 


Food and Drug Regulations 


the Sas raeney of review outcomes aid for implementing ya 


Giiteria: : 


User Fees Act 


The Department reviews pharmaceutical submissions in 
a timely manner. 


Guidance for Industry: Management of Drug Submissions, 
Health Canada 


The Department establishes quality assurance systems to Health Products and Food Branch Strategic Plan, Health 
support the consistent review of pharmaceutical submissions Canada, 2007-2012 


Therapeutic Products Directorate Strategic Plan 2006-2009, 
Health Canada 


1S09001:2008 


The Department publicly discloses information related to its Cabinet Directive on Streamlining, Our Commitment 
review of pharmaceutical submissions that have been approved, to Canadians 


rejected, or withdrawn Health Products and Food Branch Strategic Plan, Health 
Canada, 2007-2012 


The Department implements systems to manage risks to the 
review process arising from real, apparent, or potential conflicts 
of interest 


Values and Ethics Code for the Public Service, Treasury Board 


Health Products and Food Branch Strategic Plan, 
Health Canada, 2007-2012 


To determine whether Health Canada fulfilled its key responsibilities for monitoring the safety of market ed | pharmace tial, for communicat 
concerns in a timely manner, and for Le the yen ernie the ph 


Criteria 


The Department uses a risk-based approach to monitor the ¢ Federal Regulatory Post-Market Surveillance Strategy, 
safety of pharmaceuticals authorized for use in Canada Health Canada 


e Therapeutic Access Strategy, Health Canada 


The Department uses a risk-based approach to enforce industry’s | * Framework for the Management of Risk, Treasury Board 
compliance with the sections of the Food and Drug Regulations 


pertaining to pharmaceuticals ¢ Decision-Making Framework for Identifying, Assessing and 


Managing Health Risks, Health Canada 


Health Product and Food Branch’s Compliance and 
Enforcement Risk Evaluation Guide: An Approach to Decision- 
Making, Health Canada 


Food and Drug Regulations 


The Department communicates safety risks to health care Federal Regulatory Post-Market Surveillance Strategy, 
professionals and the public in a timely manner and assessed the Health Canada 
effectiveness of its communications 


Description of Current Risk Communication Documents for 
Marketed Health Products for Human Use, Health Canada 


Management reviewed and accepted the suitability of the criteria used in the audit. 


34 Chapter 4 Report of the Auditor General of Canada—Fall 2011 


REGULATING PHARMACEUTICAL DRUGS—HEALTH CANADA 


Period covered by the audit 


The period under audit was 1 January 2009 to 31 December 2010. Audit work for this chapter was 
substantially completed on 31 May 2011. 


Audit team 


Assistant Auditor General: Neil Maxwell 
Principal: Louise Dubé 


Lead Auditor: Mark Carroll 


Irene Andayo 
Daphné Lamontagne 
Margaretha Ysselstein 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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Appendix List of recommendations 


The following is a list of recommendations found in Chapter 4. The number in front of the 


recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses 


indicate the paragraphs where the topic is discussed. 


Regulating clinical trials 


4.35 Health Canada should 
strengthen its risk-based approach for 


monitoring and assessing clinical trial 
adverse drug reaction reports and for 
inspecting clinical trial sites, so 
potential safety issues are mitigated. 


(4.16-4.34) 


4.36 Health Canada should establish 
timelines for officially notifying clinical 
trial sites of non-compliant ratings and 
for reviewing proposed corrective 

measures to verify compliance with the 


Food and Drug Regulations. (4.16—4.34) 


441 Health Canada should fulfill 
long-standing commitments to enhance 
public access to information on 
authorized clinical trials, including the 
results of its clinical trial inspections. 


(4.37-4.40) 


Agreed. The Department is strengthening its risk-based 
approach to monitor and assess clinical trial adverse 

drug reaction reports and clinical trial inspections. A detailed 
standard operating procedure and strategy guide has been 
developed to prioritize the review of individual adverse drug 
reaction reports. This approach was implemented on 

4 July 2011. The Department expects to have completed a 
review of the existing risk-based process for selecting clinical trial 
inspection sites by fall 2011. This review will be used to assess 
the effectiveness of the existing process and to inform the 
development, documentation, and implementation of an 
enhanced process. 


Agreed. The Department is currently reviewing and revising its 
existing standard operating procedure for conducting clinical 
trial inspections. This revised standard operating procedure will 
emphasize establishing timelines for key steps in the inspection 
process, including notification of non-compliant ratings and the 
review of proposed corrective measures. This work will be 


completed by 31 March 2013. 


Agreed. The Department will develop policies on enhancing 
public access to information on authorized clinical trials that 
respect privacy rights and legislation. Work began in 2011 and 
will be completed by 31 March 2013. 


The Department commits to publishing periodic reports 
regarding clinical trial inspections, to provide stakeholders and 
the public with a summary view of its inspection findings 


by 31 March 2012. 
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Reviewing drug submissions 


4.53 Health Canada should ensure 
that it meets service standards for the 
review of all drug submission types—by 
giving due consideration to the 
appropriate allocation of additional 
resources from increased fees charged 
to industry, to the use of foreign 
regulatory information, and to 
streamlining its review processes. 


(4.45-4.52) 


457 Health Canada should regularly 
assess whether the procedures and 
guidelines, which were established to 
ensure timely, consistent, and high- 
quality review decisions, are interpreted 


and applied consistently by all four 
review bureaus. (4.54—4.56) 


4.63 Health Canada should disclose 
information related to new drug 
approvals in a timely manner and 
improve the transparency of “approvals 
with conditions,” rejections, and 
withdrawals of new drugs so that 
Canadians and health care professionals 
can access information about these 


drugs. (4.58-4.62) 


4.70 Health Canada should assess the 
risks posed by conflicts of interest to the 
drug review process, determine what 
measures are necessary to manage these 
risks, and implement those measures. 


(4.64-4.69) 


Fall 2011 


Agreed. Revenues from recently updated user fees will allow 
the Department to meet its well-established and internationally 
recognized performance standards. The Department will closely 
monitor its performance. It will continue to seek process 
efficiencies, such as increasing the leverage of external scientific 
expertise and the use of foreign regulatory information. The 
Department will begin piloting an approach to enhance and 
formalize the use of foreign reviews through standard operating 
procedures and guidance for industry in the fall of 2011 and will 
complete an evaluation of the pilot by 31 March 2014. 


Agreed. The Department will develop a system to regularly 
assess and ensure the use of procedures established to ensure 
timely, consistent, and high-quality review decisions 

by 31 December 2012. Implementation of the system, which will 
include assessment of compliance with procedures and 
consistency of interpretation across organization review units, 
and necessary corrective mechanisms to ensure consistent use, 


will be completed by 31 December 2013. 


Agreed. The Department will improve the transparency of 
approvals with conditions, rejections, and withdrawals to the 
Canadian public. The Department will consult with stakeholders 
in fall 2011 about expanding its public communications on post- 
approval decisions for marketed health products to include 
information on approvals with conditions, rejections, and 
withdrawals, with a view to disclosing additional information 


by June 2012. 


Agreed. The Department requires disclosure of potential 
conflicts of interest at the initiation of employment, and 
employees are subject to the Values and Ethics Code for the 
Public Service. The Department will determine if there are any 
particular risks posed by potential conflicts of interest in the drug 
review process by 31 March 2012 and, if necessary, develop and 
implement additional measures by 30 September 2012. 
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Monitoring post-market safety 


4.96 Health Canada should improve Agreed. The Department will improve the timeliness of safety 
the timeliness of safety assessments and _ assessments by fully implementing and respecting related 
the implementation of related performance standards by December 2013. This involves 


recommendations to update labels and —_ reviewing baselines for completing safety reviews. 
to issue risk communications, so 

Canadians and health care professionals 

can be informed of new drug safety 

information in a timely manner. 


(4.73-4.95) 

497 Health Canada should establisha | Agreed. The Department will establish a systematic process to 
systematic process to manage safety implement safety assessment recommendations for marketed 
assessment recommendations for drugs. The Department has already begun developing a standard 
marketed drugs, to ensure that operating procedure in this regard. The standard operating 
recommendations are dealt with procedure will include new formalized tracking systems 
appropriately and in a timely manner. (currently under development) and necessary implementation 
(4.79-4.95) procedures following a safety assessment recommendation. This 


will be in place by 31 March 2013. 


Enforcing compliance with the regulations 


4.109 Health Canada should Agreed. The Department strives to ensure that complaints 
consistently apply its risk-based received are addressed in a timely manner and are properly 
standard operating procedures, so the documented. The Department will review its processes to ensure 
priority of the drug complaints it better documentation, conduct training sessions, and implement 
receives is properly documented and more robust performance monitoring by 31 March 2013. 


addressed in a timely manner. 


(4.100-4. 108) 
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Maintaining and Repairing Military 
Equipment—National Defence 


Performance audit reports 


This report presents the results of a performance audit conducted by the Office of 
the Auditor General of Canada under the authority of the Auditor General Act. 


A performance audit is an independent, objective, and systematic assessment 

of how well government is managing its activities, responsibilities, and resources. 
Audit topics are selected based on their significance. While the Office may 
comment on policy implementation in a performance audit, it does not comment 
on the merits of a policy. 


Performance audits are planned, performed, and reported in accordance with 
professional auditing standards and Office policies. They are conducted by 
qualified auditors who 


* establish audit objectives and criteria for the assessment of performance; 
* gather the evidence necessary to assess performance against the criteria; 
* report both positive and negative findings; 

* conclude against the established audit objectives; and 


* make recommendations for improvement when there are significant 
differences between criteria and assessed performance. 


Performance audits contribute to a public service that is ethical and effective 
and a government that is accountable to Parliament and Canadians. 


Table of Contents 


Main Points 


Introduction 


Maintenance and repair activities, responsibilities, and budgets 
Findings from previous audit 

Changing approaches to maintenance and repair 

Focus of the audit 


Observations and Recommendations 


Allocation and monitoring of financial resources 


Financial resources for maintenance and repair activities were allocated to priorities 
Total cost information on maintenance and repair is missing 
National Defence needs better performance information 


Contracting approaches for existing equipment 


The Optimized Weapon System Management program was designed to address key risks 


Implementing new contracting approaches for existing equipment has been slower 
and more limited than intended 


Demonstrating cost savings is a challenge for the Optimized Weapon System Management program 


Contracting approaches for new equipment 


While In-Service Support Contracting Framework’s risks have been identified, 
mitigation measures are limited 


Insufficient resources and oversight threaten implementation of the In-Service Support 
Contracting Framework 
Coordination with other federal departments and Canadian industry needs to be strengthened 


Conclusion 
About the Audit 


Appendix 


List of recommendations 
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What we examined 


Why it’s important 
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Main Points 


National Defence and the Canadian Forces own, operate, and are 
responsible for maintaining and repairing military aircraft, ships, and 
land vehicles costing more than $30 billion. In 2009-10, National 
Defence spent more than $2 billion to maintain and repair its military 
equipment. This included expenses for routine inspections, preventive 
maintenance, corrective repairs, spare parts supply, periodic repair and 
overhaul, engineering changes, and other related tasks. 


Thousands of personnel are engaged in maintenance and repair 
activities, which include everything from turning wrenches on bases or 
while deployed on missions to engineering, logistics and spare parts 
management, training, contracting, and administrative support. 


We examined how National Defence allocates and manages financial 
resources for the maintenance and repair of its military equipment. 
We also examined its approaches to contracting for maintenance and 
repair services. 


Audit work for this chapter was substantially completed 


on 30 April 2011. 


To undertake training and to meet assigned missions safely and 
successfully, military equipment must be kept in good working 
condition and be ready for action on short notice. How National 
and the reliability of the 


Defence allocates the funds available 
information it uses to support short- and long-term decisions—is 
critical to the ability and readiness of the Canadian Forces to meet 


their assigned missions. 


Military spending on maintenance and repair also makes a significant 
contribution to the Canadian economy every year. Many Canadian 
companies depend on it for a portion of their business. According to 
the 2008 Canada First Defence Strategy, the government plans to 
spend $60 billion on new military equipment over 20 years and 


$140 billion for spare parts, maintenance, and training. 
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Over the last decade, National Defence has made sweeping changes in 
its approach to contracting for maintenance and repair of both existing 
and new equipment. The changes transfer much of the responsibility 
to the private sector, with significant implications for National Defence 
and Canada’s defence industry. The new contracting approach for 
existing equipment was intended to reduce the Department’s contract 
management activities and costs by bundling hundreds of short-term 
maintenance contracts into a few longer-term contracts. The approach 
for new equipment goes even further, awarding both the acquisition 
and the long-term maintenance and repair contracts to the original 
equipment manufacturer or supplier. The Department has identified 
significant risks in this approach, including limited flexibility if 
requirements change over the life of the equipment, dwindling 
maintenance and repair skills and expertise in the Canadian Forces, 
and total dependence on one supplier for each fleet. 


What we found == - Overall, National Defence has planned and managed the 
maintenance and repair of military equipment to meet operational 
priorities in the short term. The annual process of allocating 
available funds provides an effective forum to discuss priorities, 
with wide participation of those responsible for maintaining and 
repairing military equipment and those who need it for operations 
and training. 


e 


National Defence’s ability to meet training and operational 
requirements over the long term is at risk due to weaknesses 

in implementation and oversight of its contracting approaches 

for maintenance and repair, deficient management information 
systems, and the lack of sufficient cost and performance information. 


: The Department has not taken the actions or provided the central 
resources and oversight required to support the implementation of its 
new contracting approaches successfully. The lack of concerted action 
and follow-through on the new contracting approach for existing 
military equipment has resulted in slower and more limited 
implementation than planned. As a consequence, National Defence 
has lost opportunities to derive the potential benefits of improved 
performance, improved accountability, and reduced costs. In addition, 
National Defence is not adequately monitoring and mitigating the 
significant risks created by its approach for new equipment. 


- There are long-standing deficiencies in information management 
systems used to support decision-making for maintenance and 
repair activities, first raised by us in a 2001 audit. As a result, 
National Defence lacks complete, reliable, and integrated 
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information on the total actual costs of maintenance and repair, 
because some of the costs 


salaries and infrastructure, for 


example—are not captured in its asset management information 
systems. The absence of this information impedes its ability to make 
informed decisions about the life-cycle management of its fleets or to 
determine whether it is putting enough funds each year into 
maintenance and repair. In 2001, National Defence expected to fully 
implement an integrated asset management system by 2004. The 
Department now expects a new system to be introduced on all 
Canadian Forces bases by mid-2012. Fully implementing this new 
system will likely take many years. 


There is a significant gap between the demand for maintenance and 
repair services and the funds made available. In addition, National 
Defence has indicated it is likely that its long-term investment plan 
for new equipment has allocated insufficient funds for equipment 
life-cycle costs. Although National Defence knows that postponing 
maintenance and repair tasks creates future risks—such as reduced 
availability of equipment, more laborious and expensive repairs, and 
reduced life expectancy of military equipment—the Department 
does not regularly monitor these impacts. Consequently, it does not 
know the specific long-term impacts of the funding gap on operations 
and training activities. 


The Department has responded. The Department agrees with 
all of our recommendations. Its detailed responses follow the 
recommendations throughout the chapter. 
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Military equipment—tn this report, military 
equipment refers to ships, submarines, 
airplanes, helicopters, and land vehicles 
(armoured or not, wheeled or tracked) used by 
the Canadian Forces for training and 
operations. This equipment is also sometimes 
referred to as weapon systems. Each 
equipment or weapon system is made up of a 
number of subsystems (engines, radars, guns, 
radios, etc.), which consist of different parts. 
Military equipment is managed as a fleet when 
there are many ships, aircraft, or vehicles of 
the same model. 


Readiness—A measure of the ability of a 
Canadian Forces unit to undertake an approved 
task. Readiness includes several aspects, 
including personnel, training, and equipment. 
National Defence establishes readiness 
standards or targets for most of its military 
equipment. This chapter focuses on equipment. 
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Introduction 


9.1 The Canada First Defence Strategy, released by the government 
in 2008, defines six core missions that the Canadian Forces must be 
prepared to undertake in Canada or abroad. Based on these missions, 
National Defence determines the capabilities that it needs to develop 
and sustain. To a large extent, sustaining these capabilities depends 
on military equipment being kept in good working condition and at 
a certain level of readiness. Effectively planning, supporting, and 
executing maintenance and repair activities for military equipment 

is therefore crucial to the Canadian Forces’ ability to meet their 

core missions. 


§.2 National Defence and the Canadian Forces own, operate, and are 
responsible for maintaining and repairing military equipment costing 
more than $30 billion that is used for domestic and international 
operations and training. This military equipment includes 4 submarines, 
more than 30 large ships, over 350 aircraft, and about 9,000 military 
land vehicles. Maintaining and repairing all this equipment involves 
thousands of civilian and military personnel and many private sector 
firms in a diverse and often complex set of activities. 


5.3. Military spending on maintenance and repair makes a significant 
contribution to the Canadian economy every year, and many Canadian 
companies depend on it for a portion of their business. If current trends 
continue, this contribution could increase in the future as more and 
more responsibility for providing maintenance and repair services is 
transferred to private sector firms in Canada and abroad. As announced 
in the 2008 Canada First Defence Strategy, the government plans to 
spend $60 billion on new military equipment and $140 billion on spare 
parts, maintenance, and training over 20 years. 


Maintenance and repair activities, responsibilities, and budgets 


5.4 At National Defence, maintenance is a preventive, scheduled 
activity that is intended to reduce the probability of equipment failure 
and extend the life of the asset. Repair is an activity that restores an 
item to serviceable condition by correcting faults or replacing 
unserviceable pieces with new, overhauled, rebuilt, or reconditioned 
components. Repairs are unpredictable and are thus more difficult to 


plan for and forecast than maintenance activities. 


5.5 Inthe 2009-10 fiscal year, National Defence estimated that it 
spent just over $2 billion to maintain and repair its military equipment. 


This included expenses for routine inspections, preventive 
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maintenance, corrective repairs, spare parts supply, periodic repair and 
overhaul, retrofits, engineering changes, preparation for special 
operations, and other related tasks. This amount did not include the 
salaries of personnel or the costs of infrastructure used to maintain and 
repair military equipment. The full spending on maintenance and 
repair activities is thus greater than $2 billion. 


5.6 The useful life of military equipment can sometimes extend more 
than 40 years from initial acquisition to final disposal. Maintenance 
and repair activities are categorized by National Defence into 

four different lines of work, based on their level of complexity and the 
time required to complete them. These activities can be performed by 
Canadian Forces technicians or by private sector firms: 


¢ Simple and short-term preventive maintenance and minor 
repair activities (first- and second-line) are carried out relatively 
frequently by civilians and military personnel on bases across 
the country or in the field. These tasks are typically completed 
within 24 hours. 


¢ Lengthier and more complex inspection, major repair, or 
complete equipment overhaul activities (third- and fourth-line) 
are centrally managed by the Assistant Deputy Minister (ADM) 
for Materiel, and are executed by the Department or by the 
private sector in specialized facilities. These tasks can take days, 
weeks, or months to complete. 


5.7 These maintenance and repair activities are supported by 
equipment program management divisions that report to the ADM 
for Materiel. These divisions are responsible for the life-cycle 
management of aircraft, ships, and land vehicles; they identify 
maintenance and repair needs for each fleet, provide engineering and 
technical support, manage upgrade programs, and ensure that 
sufficient stocks of spare parts are purchased and made available to 
maintenance and repair technicians on a timely basis. Considerable 
variations exist in the specific ways that the Army, Air Force, and Navy 
carry out maintenance and repair for their equipment. 


Findings from previous audit 


5.8 We reported on National Defence’s maintenance and repair 
activities in our 2001 chapter National Defence—In-Service 
Equipment. At the time, we observed that, in general, the information 
the Department needed to manage its maintenance and repair 
activities was often unavailable, incomplete, inadequate, or inaccurate. 
We also noted that budget restrictions, shortages of qualified 
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maintenance personnel, problems with spare parts supply, and aging 
equipment were having detrimental impacts on the Canadian Forces’ 
ability to meet equipment readiness standards. Some training exercises 
and deployments had been affected by the unavailability of military 
equipment. 


Changing approaches to maintenance and repair 


§.9 Since our previous audit, National Defence has faced a number 
of circumstances that have changed the context in which maintenance 
and repair activities take place. 


* Increased pace of operations. Canadian Forces have been 
deployed in Afghanistan since 2002, resulting in an increased 
pace of operations in the Army and the Air Force. This effort has 
accelerated the usage rates for various land vehicles and aircraft, 
increasing the need for maintenance and repair. In addition, 
deployment of qualified maintenance technicians to areas of 
operation abroad has meant that Army and Air Force units in 
Canada have had fewer technicians available to carry out 
maintenance and repair work. 


* Human resources challenges. For many years, the Department 
has faced shortages of skilled employees in some maintenance and 
repair trades. In addition, cuts made in the 1990s to National 
Defence human resources have resulted in fewer employees who 
are responsible for planning, managing, and contracting 
maintenance and repair activities. The looming retirement of 
baby boomers and ongoing competition with the private sector for 
qualified personnel are further challenges in this area. 


¢ Increasing cost. As they age, the cost of maintaining, upgrading, 
and repairing existing military fleets increases. In addition, 
modern replacements for old fleets usually carry more 
technologically advanced systems that cost more to maintain than 
the ones they are replacing. Total maintenance and repair costs 
over the complete life cycle of military equipment are often two or 
three times greater than the equipment acquisition cost. 


5.10 In response to these and other challenges, over the past decade 
National Defence has explored new ways of meeting its maintenance 
and repair needs. In 2002, National Defence introduced the Optimized 
Weapon System Management (OWSM) program for existing military 
equipment and, in 2008, the In-Service Support Contracting 
Framework (ISSCF) for new equipment acquisitions. Consistent with 
practices in other countries, National Defence is increasing its use of 
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Allocation and monitoring 
of financial resources 


long-term, performance-based contracts with original equipment 
suppliers and/or specialized private sector firms. These new contracting 
approaches have meant sweeping changes for National Defence and its 
personnel, for federal departments (such as Public Works and 
Government Services Canada), and for the Canadian defence industry. 


Focus of the audit 


§.11 This audit examined whether National Defence planned and 
managed the maintenance and repair of its military aircraft, ships, and 
land vehicles to meet its operational and training requirements. More 
specifically, we examined whether the Department had 


* appropriately allocated and monitored financial resources for the 
maintenance and repair of its military equipment to meet training 
and operational requirements; and 


* appropriately established and implemented maintenance and 
repair contracting approaches for its military aircraft, ships, 
and land vehicles. 


§.12 Our examination of the management of the various maintenance 
and repair funds covered the planning cycles for the 2009-10 

and 2010-11 fiscal years. Our examination of contracting approaches 
covered the decade that has elapsed since our 2001 audit, because 
work on many of the long-term maintenance contracts that we looked 
at started in the early 2000s and is ongoing today. 


5.13 More details about the audit objectives, scope, approach, 
and criteria are in About the Audit at the end of this chapter. 


Observations and Recommendations 


5.14 The ultimate purpose of maintenance and repair activities is 

to ensure that military equipment meets operational requirements in 
a cost-effective manner over its intended useful life. If adequate 
maintenance cannot be sustained, then, over time, a fleet can go from 
being very reliable to one that is struggling to meet operational needs. 
We examined whether National Defence appropriately allocated 

and monitored financial resources to maintain and repair its military 
aircraft, ships, and land vehicles to meet operational and 

training requirements. 


5.15 Funds to maintain and repair National Defence’s military aircraft, 
ships, and land vehicles come from different budgets within the 
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Department. The largest is the National Procurement (NP) budget, 
managed by the Assistant Deputy Minister (Materiel). This budget 
provides for the purchase of spare parts, maintenance equipment, and 
contracted services for third- and fourth-line activities, as well as limited 
first- and second-line activities. Although portions of the overall NP 
budget are allocated to the procurement of ammunition, common 
equipment such as clothing, and other miscellaneous purchases, this 
chapter is strictly concerned with the portion allocated to maintenance 
and repair activities. In the 2009-10 fiscal year, about $1.8 billion of the 
total NP budget was spent on maintenance and repair activities. The 
total amount allocated by the Department to the NP budget is not fixed. 
The amount specifically allocated for maintenance and repair fluctuates 
year to year. Exhibit 5.1 presents the NP allocations for the 2006-07 

to 2010-11 fiscal years. 


5.16 Over the past several years, funds allocated by National Defence 
did not cover the expressed demand for maintenance and repair 
activities. The size of the gap between demand and NP funding has 
been relatively consistent in recent years. The government noted in its 
2008 Canada First Defence Strategy that the financial resources 
allocated to the NP budget covered only 70 percent of demand in 
recent years, significantly impeding the Canadian Forces’ ability to 
train personnel and to maintain high readiness levels. 


Exhibit 5.1 Allocated funds have not covered the cost of demands for maintenance and repair 
activities for the past five fiscal years 


$ billions 
3.0 


2.5 


2.0 
National Procurement 


15 a Demand 


[] Allocated funds 


0.5 


0.0 
2006-07 2007-08 2008-09 2009-10 2010-11 


Fiscal years 


Source: Prepared based on National Defence documents 
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Financial resources for maintenance and repair activities were allocated 
to priorities 


§.17. According to the Treasury Board of Canada Secretariat Guide 
to Management of Materiel (2008), effective planning and resource 
allocation are essential for delivering a program, for achieving value 
for money, and for sound stewardship. In this context, we examined 
whether the Army, Navy, and Air Force equipment program 
management divisions had established priorities for maintenance and 
repair of their military equipment, and whether National Defence had 
allocated financial resources based on these priorities. 


5.18 The annual planning cycle for the NP budget is an elaborate 
process that involves many individuals and different organizations 
within the Department, under the leadership of the Materiel Group. 
The objective of this process is to allocate available funds among the 
different maintenance and repair activities. At the beginning of the 
planning cycle, the equipment program management divisions of the 
Air Force, Army, and Navy are notified of a preliminary amount of 
NP funding (known as the planning allocation), which they 
subsequently allocate to their various fleets. Specific needs and their 
level of priority are then reviewed by various planning committees, and 
included in a business plan. These business plans provide estimated 
cost demands for each priority, expressed by fleet. The demands are 
based on historical usage, planned usage, and known and anticipated 
special requirements. 


5.19 The business plans feed into the deliberations of the National 
Procurement Oversight Committee. The Committee’s mandate is to 
provide strategic advice and make recommendations to the National 
Defence Program Management Board—a committee of the most 
senior managers who provide advice and guidance on resource 
management—on major NP activities, plans, and policies and to 
facilitate their implementation. The final decisions on NP allocations 
for the next fiscal year are taken by the Defence Finance Committee, 
an advisory body that provides high-level strategic financial guidance 
to the Deputy Minister. 


5.20 Overall, in our view, the NP planning process provides an effective 
forum for discussing needs and priorities with a wide range of personnel 
responsible for maintaining and repairing military equipment, as well as 
those who use such equipment for operational and training purposes. We 
found that the equipment program management divisions of the Army, 
Navy, and Air Force had each defined their maintenance and repair 
priorities in their respective business plans for the 2010-11 fiscal year. 
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The final allocation decisions generally reflected these priorities. The 
business plans also identified the potential impacts of not funding their 
priority activities, such as reduced purchase of spare parts, deferred 
maintenance and repair activities, and reduced operational output (for 
example, flying hours). 


5.21 During the year, circumstances can change, and some planned 
activities may be either postponed or cancelled. In response, financial 
resources must be reallocated and used before the end of the fiscal year; 
otherwise, they will lapse. We observed that processes are in place to 
reallocate funds that become available during the year. A review takes 
place regularly to assess the availability of funds, and to identify funds 
that are unlikely to be spent by year-end. In these cases, a decision can 
be made to cancel or to postpone an activity to a future year, and to 
reallocate the funds to another current-year activity. When funds cannot 
be spent before the end of the fiscal year, managers try to reallocate to 
other priority activities that can be rapidly undertaken before year-end. 
However, the reallocation of funds does not always result in complete 
spending by year-end. For the 2010-11 fiscal year, National Defence 
estimated that it was unable to spend $193 million of NP funding 
dedicated to maintenance and repair. 


Total cost information on maintenance and repair is missing 


§.22 The Treasury Board Policy on Management of Materiel (2006) 
requires that a materiel management information system be in place to 
collect and generate complete and accurate data on materiel assets. 
Such a management information system should be integrated with 
departmental financial information systems, and should support timely 
and informed materiel management decisions. We examined whether 
National Defence has an integrated asset management system to 
collect and use financial information required to support resource 


allocation decisions for its maintenance and repair activities. 


§.23 To support informed decisions about where to allocate overall 
financial resources for maintenance and repair activities, National 
Defence must have, among other things, reliable information on the 
total costs of these activities. For example, having total cost 
information can help the Department determine whether it would be 
more cost-effective to assign maintenance and repair tasks for a given 
fleet to the private sector, rather than to retain the required in-house 
capacity. Having total cost information on specific fleets can also 
enable the Department to determine when it will become more cost- 
effective to replace a fleet, rather than to continue to maintain it. The 
need for reliable total cost information is heightened by the fact that 
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National Defence has indicated that it is likely that its long-term 
investment plan for new equipment has allocated insufficient funds for 
equipment life-cycle costs. 


5.24 In general, assessing the total costs of maintenance and repair 
activities for the Department as a whole or for individual military fleets 
involves determining the costs of 


¢ the salaries of all personnel (civilians, regular forces, and reserve 
forces) involved in planning and executing maintenance and 
repair; 


* contracted services (which may include provision of spare parts); 


* acquiring spare parts through the central Canadian Forces supply 
system or locally, through a base’s budget; 


* maintaining the required facilities; and 


* specialized maintenance and test equipment. 


§.25 While National Defence has tools to estimate the costs of 
maintenance and repair activities, we found that it does not track the 
total actual costs of these activities for the Department as a whole or 
for specific fleets. These costs cannot be readily determined, to a large 
extent because the Department has not yet put in place adequate 
information systems. For example, the salaries of most personnel are 
not captured by the Department’s various current asset management 
information systems. Other maintenance and repair costs, like those 
related to infrastructure, are also not captured. 


5.26 In addition to the NP budget, each Canadian Forces base 
dedicates a portion of its operations and maintenance budget to 
support first- and second-line maintenance and repair activities. These 
base expenditures add up to several hundred million dollars per year. 
However, because base-level maintenance and repair expenditures are 
not accounted for by fleet, exact figures are not available. 


5.27 The National Procurement Oversight Committee has 
acknowledged the lack of good financial information for decision 
making. This issue is not new. In our 2001 audit, we concluded that 
National Defence needed to improve its management information 
systems, and the quality of data they contained. The Materiel 
Acquisition and Support Information System (MASIS) project, 
intended to fill this need, was launched in September 1999. At the 
time of our 2001 audit, National Defence had expected to fully 
implement MASIS across the Canadian Forces by 2004, and had 
indicated that it had made this implementation a priority. 
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9.28 By April 2010, MASIS had not been fully implemented across 
the Canadian Forces. It was then merged with the Department's 
Financial Management Accounting System (FMAS) to become the 
Defence Resource Management Information System (DRMIS). 
Although this merging represents an important step toward 
implementing an integrated asset management information system, its 
full potential will not be realized until all Canadian Forces bases have 
transitioned to the new system, personnel have been adequately 
trained, and analytical tools to derive useful trend information have 
been implemented. 


9.29 National Defence now expects the DRMIS system to be 
introduced on all Canadian Forces bases by mid-2012. Based on the 
experience with MASIS, in our opinion, fully implementing DRMIS 
will likely take many years and will require sustained efforts on a 
priority basis. In the meantime, in our opinion, the absence of 
complete, actual, and reliable overall and fleet-specific cost 
information impedes National Defence’s ability to make informed 
decisions regarding the allocation of funds for the maintenance and 
repair of its military equipment or to analyze options related to the life- 
cycle management of its fleets. Ultimately, the Department does not 
have a firm basis on which to determine whether it is putting enough 
funding into maintenance and_repair activities each year. In our view, 
this lack of information, coupled with the funding gap, creates a risk 
that, over time, the Canadian Forces may not be able to maintain all of 
its current capabilities and therefore may be limited in the size and 
variety of missions it can undertake. 


5.30 Recommendation. National Defence should ensure that it 
develops the ability to produce overall and fleet-specific total cost 
information for its maintenance and repair activities. These costs 
should include, at a minimum, expenses related to personnel, 
contracted services, spare parts, maintenance equipment, and 


infrastructure. 


The Department’s response. Agreed. Project Management Office 
(PMO) MASIS is rolling out Systems, Applications, and Products 
(SAP) in support of maintenance of weapon systems and equipment. 
The SAP system, called the Defence Resource Management 
Information System (DRMIS), is currently in place for over 90% of the 
Army and Navy, and approximately 10% of the Air Force. There are 
plans to implement DRMIS in the remainder of the Canadian Forces 
and this system has the capacity to meet the information requirements 
identified in this recommendation. This is planned for completion by 


December 2013. 
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The MASIS Phase V on-going project has integrated spares inventory 
with procurement, financial and maintenance engineering data under 
a single platform. This integration paves the way to equipment, fleet 
and overall cost reporting. As the implementation of MASIS 
progresses, personnel, spare parts and procurement costs are being 
captured. At this time, infrastructure cost is not captured in MASIS, 
but will be in the future as the Department continues to advance its 
Enterprise Resource Planning strategy and its SAP implementation. 


In parallel, the Repair and Overhaul business process involving the 
management of the repair line will be assessed against best industry 
practices to determine the improvements required to optimize this part 
of the supply chain. This will take place prior to MASIS Phase V 
blueprinting for repairs and overhaul scheduled in the fall of 2011. 


The Maritime equipment program management division is actively 
moving towards a class-focused program that will improve the ability 
to capture total maintenance and repair costs for each class of naval 
platform. 


Finally, the work being conducted in developing costing tools and 
guidelines in response to the Auditor General’s audit report on the 
Acquisition of Military Helicopters will also assist in meeting this 
recommendation. 


National Defence needs better performance information 


5.31 The Treasury Board Policy on Management of Materiel (2006) 
states that departments should ensure that their materiel management 
framework provides them with relevant performance information, and 
supports informed decision making. We examined whether National 
Defence collected and used the performance information it requires to 
support financial resource allocation decisions for its maintenance and 
repair activities. 


5.32 National Defence does gather some information about the 
utilization rates or state of its military equipment. For example, the 
Air Force gathers data on the number of hours of flying per year, 
the Navy collects data on days at sea and state of readiness of each 
ship, and the Army measures the percentage of vehicles that are 
not serviceable. 
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5.33 The stated purpose of NP expenditures is to keep military 
equipment at a planned state of readiness. The primary indicator that 
National Defence uses to measure the performance of the NP 
allocations is the actual readiness status—that is, the current state of 
an item of equipment or a fleet measured against its expected readiness 
level. In our opinion, however, readiness status is not a meaningful or 
sufficient performance indicator for three reasons. First, quantifiable 
readiness targets do not exist for all fleets of military equipment. Also, 
whether or not readiness targets are being met is influenced by factors 
other than NP funding, such as the availability of personnel. Most 
importantly, readiness targets for many fleets are based on existing 
capacity and capability; they have been developed on the basis of what 
is achievable, not what is needed to meet expected mission and 
training requirements. In practice, over time, some readiness targets 
have been downgraded to meet the capacity that is affordable under 
available funds. In effect, this means that NP funding will generally be 
sufficient to meet readiness targets, because the targets are related to 


the availability of funds. 


9.34 For maintenance and repair work that is contracted to 
private sector firms, National Defence sets and measures performance 
indicators, such as turn-around time for repairs, the availability 

of spare parts, and the reliability of repairs. In our view, these are 
the types of measures that National Defence could apply in its own 
operations. Although the Department has recognized the need 

to systematically measure the efficiency and effectiveness of its own 
equipment management activities, it does not currently do so. 

The Navy’s equipment program management division is now 
developing a performance management framework that aims 

to measure support activities, such as provision of spare parts or 
engineering changes. The Navy expects the framework to help 


* establish the link between maintenance and repair activities 
and the impact on the state of its fleets, and 


* predict the future state of its fleets. 


5.35 The above means that National Defence has limited capacity 
to assess the impact of its annual NP allocation decisions. Together 
with the lack of total cost information, this means that the 
Department does not have a firm basis for determining whether 
the same result could be achieved at less cost or if the same 


expenditure could achieve better results. 


— - — 
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Availability—The proportion of time that a 
fleet of military equipment is in an operable 
state (not undergoing maintenance) in relation 
to the total operational time available during a 
stated period. 


5.36 National Defence is aware that postponing maintenance and 
repair tasks has immediate consequences, and also creates the risk of 
future impacts. In 2008, it undertook a one-time study of the impact of 
the NP funding gap in previous years, which demonstrated that it has 
had serious impacts on the Air Force, Army, and Navy, such as 


¢ backlogs of required work, 
* fewer sea days and flying hours, 


¢ reduced availability of equipment for training activities and 
operational requirements, 


* more laborious and expensive corrective repairs, and 


* reduced life expectancy of military equipment. 


However, National Defence does not regularly conduct such studies 
and does not regularly monitor these impacts; consequently, it does not 
know the specific long-term impacts created by its estimated funding 
gap on operations and training activities. 


9.37 Recommendation. National Defence should develop and 
implement a capacity to provide information on the performance and 
impacts of maintenance and repair activities, their effectiveness, and 
their efficiency within each of the Army, Navy, and Air Force, using 
common performance measures where possible. 


The Department’s response. Agreed. As noted in the previous 
response, PMO MASS is rolling out the Defence Resource 
Management Information System (DRMIS) in support of maintenance 
of weapon systems and equipment, which will have the capacity to 
provide the information outlined in this recommendation. This 
implementation is planned for completion by December 2013. 


Business Intelligence specific queries made available through MASIS 
Phase V enable some searches on equipment and environment-specific 
performance and measures. Further development is required to 
introduce advanced queries and tools to enhance this Department’s 
ability to perform forward planning so that overall effectiveness and 
efficiency are improved. The Department is also pursuing work on the 
development of performance measures in the area of inventory 
management. 
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9.38 ‘Treasury Board policies on risk management and management of 
materiel require federal departments to assess the risks to which their 
assets and program activities are exposed, and to implement cost- 
effective measures to control those risks. We examined whether 
National Defence had designed its contracting approaches to address 
the key risks to which its military equipment and its maintenance and 
repair activities were exposed, and whether it had appropriately 
implemented these approaches. 


The Optimized Weapon System Management program was designed to address 
key risks 


5.39 National Defence has historically relied on the private sector to 
maintain and repair various components and systems of its military 
aircraft, ships, and land vehicles, especially for third- and fourth-line 
activities. This work has generally been managed with contracts to 
private sector firms that get paid for the time and materials needed to 
complete prescribed inspections and repairs; this approach is known as 
the traditional approach. Because complex military equipment can 
have thousands of components, hundreds of contracts were sometimes 
required to maintain a single fleet. In the 1990s, the Department was 
managing thousands of these “time and materials” contracts every year. 
Following significant staff cuts in the mid-1990s, National Defence 
recognized that its traditional approach to contracting would no longer 
be sustainable, because it would not have the personnel needed to 
manage all the contracts. 


5.40 During the same period, National Defence recognized that the 
traditional approach had often resulted in unsatisfactory performance 
and poor contractor accountability, which put at risk the availability of 
military equipment. For example, several contracted aerospace firms 
were often late in completing aircraft maintenance and repair 
activities, which diminished the availability of the aircraft for 
operations and training activities. 


§.41 As it looked for solutions to these problems, the Department 
became interested in new contracting approaches employed by the 
militaries in the United States, the United Kingdom, and Australia to 
reduce costs and improve performance. Known as “performance-based 
logistics” and sometimes called “contracting for availability,” these 
approaches set measurable performance targets (such as turn-around 
time for repairs, quality of work performed, and availability of spare 
parts) and provide financial incentives for private sector firms to meet 


or surpass these targets. 
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CC-130 Hercules 


CP-140 Aurora 


5.42 In 1998, the Air Force took the lead in designing a new 
contracting strategy for its main existing fleets. This effort led to the 
adoption of the Optimized Weapon System Management (OWSM) 
approach in 2002, with the intent to 


* move from having many short-term contracts to fewer long-term 
contracts, 


* decrease the number of civilian and military personnel needed to 
manage and execute maintenance and repair activities, 


* transfer more responsibility to the private sector, and 


* establish performance objectives for private sector firms and 
provide incentives to improve performance and reduce costs. 


5.43 Exhibit 5.2 presents OWSM’s main characteristics, as compared 
to the traditional approach. The Aerospace Equipment Program 
Management Division (Air Force) under the Assistant Deputy 
Minister (ADM) for Materiel was responsible for the development of 
the OWSM program. By bundling hundreds of traditional 
maintenance contracts under only a few OWSM contracts, and by 
offering incentives to private sector firms, the Department expected to 
improve the firms’ accountability for performance. The OWSM 
directive also required internal project teams to generate 15 percent 
cost savings compared to the annual fleet support costs of their 
previous maintenance approach. Overall, we found that the OWSM 
contracting approach was designed to address the key pressures and 
risks facing the Air Force’s maintenance and repair activities. 


ek SI WEE 2 OR a ee eee eS 
Exhibit 5.2 Two new contracting approaches have introduced significant changes within National Defence 


Applies to : a Many existing fleets Several existing fleets All new fleets 
Number of contracts. : | Hundreds of support contracts 1 to 5 support contracts 1 support contract per fleet, 
per fleet per fleet established at fleet acquisition 

Level of application | Component level (e.g. radar) Major system level (e.g. avionics) | Fleet level (e.g. aircraft) 

Contract type Time and materials Performance-based, with Performance-based, with 
incentives; fixed price where incentives; fixed-price 
possible 

Contract term 1 year 5+ years 20+ years 

Required management a Large departmental Moderate departmental Smaller departmental 

resources management staff required management staff required management staff required 


Source: National Defence 
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Implementing new contracting approaches for existing equipment has been slower 
and more limited than intended 


9.44 In 2002, the OWSM program was to be applied to four Air Force 
fleets that could benefit from the new approach: they had high annual 
maintenance and repair costs and enough years of useful life left to 
justify the initial investments in the program. These fleets were the 
Hercules transport airplane, the Aurora maritime patrol airplane, 

the Griffon helicopter, and the CF-18 jet fighter. The objective was 
to put in place by December 2005 nine contracts to support these 
four fleets. We examined whether National Defence had met its target 
dates for the establishment of these contracts. Where contracts were 
signed, we examined whether they included the required elements, 

as set out in internal guidance. 


5.45 Three of the nine planned contracts were awarded by the 
original target date of December 2005 (Exhibit 5.3). According to 
National Defence, these OWSM contracts have led to performance 


CF-18 Hornet 


improvements, such as faster maintenance turn-around times, increased 
aircraft availability, and improved management of spare parts. Two of the 
nine planned contracts were awarded in late 2010 and early 2011. As of 
April 2011, the remaining four contracts had yet to be awarded. 


Exhibit 5.3 Only three of nine contracts were awarded by the target date 


ate Contract awarded by target | Contract awarded after target 
ME | date of December 2005: | — date of December 2005: Contracts still not awarded 
, Aircraft” —-— dateofaward = | ~———sdatte of award as of 30 April 2011 
) Airframe October 2005 
~ + 
CC-130 Hercules Avionics Contract pending 
Engines* Contract pending 
Airframe Contract pending 
= — _ _ —— | BN —s —— 4 a = 
CF-18 Hornet . Avionics October 2010 
= ————~—-— - -— -—— — +— -- -— + —-- —— —_—_— —— —_— 
Engines | Contract pending 
Airframe | June 2005 / 
} . — — + —______— $$$ — +— 
CP-140 Aurora Avionics June 2005 | 
— —+- — : _ ————EE — — + 
Engines* Contract pending 
CH-146 Griffon All systems January 2011 : 
Total 5 Bee FER ROIS A Sian SRI PvE “f 
*The CC-130 Hercules and the CP-140 Aurora use the same engine model. A single engine maintenance contract is being sought for both fleets 
Source: Prepared based on National Defence documents 


Report of the Auditor General of Canada—Fall 2011 Chapter 5 19 


MAINTAINING AND REPAIRING MILITARY EQUIPMENT—NATIONAL DEFENCE 


Airframe—the structural components of an 
aircraft, such as fuselage, empennage, wings, 
and landing gear, but excluding propulsion 
(engines, rotor, or reactors), electronics 
systems, and any mounted weapon. 


§.46 Although the five contracts that we reviewed generally included 
the expected OWSM contract elements, we found that the incentive 
and penalty clauses called for in the OWSM framework have so far 
been fully implemented only in the case of the Hercules airframe 
contract. Since the incentives in OWSM contracts are intended to 
drive better performance, which translates into increased availability 
of fleets, delays in their implementation mean that the expected 
program benefits will not be realized to their full extent. 


5.47 The Air Force established a board of directors to review progress 
on OWSM program implementation and related organizational changes. 
Housed within the Aerospace Equipment Program Management 
Division, the Board started operating in 2003 and met regularly in 2003, 
2004, and 2005 to review progress and provide management attention 
and oversight. Meetings became less frequent starting in 2006, following 
the departure of the program leader, and the last one was held in 
February 2009. As a result, oversight by the Board has diminished. 


5.48 Over the years, National Defence has identified several challenges 
that contributed to delays in the OWSM program implementation: 


* Resistance to change. The OWSM approach entailed 
significant changes in the roles and responsibilities of officials 
working in equipment program management divisions. The 
required culture change has faced resistance and has been slow 
to happen. 


* Lack of capacity. OWSM implementation required new, 
specialized skills for contract management that often did not exist 
in project teams. Training to develop the required skills was 
insufficient, and the high turn-over rate of military personnel 
did not favour skill retention within project teams. In addition, 
personnel numbers were often insufficient to undertake 
planned tasks. 


* Approval processes. Completing the work necessary to seek 
approval from the Treasury Board of Canada sometimes took 
longer than planned. In 2009, National Defence was required to 
complete a study of the potential impacts on the projected CF-18 
OWSM contract on Canadian small and medium enterprises. 
National Defence did not seek contract approvals until the study 
was completed in 2010. 


* Negotiations with private sector firms. The OWSM approach was 
new for private sector firms, and they had to adapt to it, too. In 
certain cases, firms have been reluctant to adopt the new approach, 
and this reluctance has made negotiations more difficult. 
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9.49 We are concerned that a lack of capacity within the Aerospace 
Equipment Program Management Division and the reduced activity of 
the OWSM Board of Directors in recent years mean that current and 
planned OWSM contracts will not be completed and implemented as 
expected. Given that the Aurora, Hercules, and CF-18 fleets are all 
scheduled to be retired between 2014 and 2020, only a few years 
remain for the Air Force to derive benefits from the implementation of 
these contracts. 


§.50 Of greater concern is the lack of concerted action across 

the Canadian Forces to implement new contracting approaches 

for existing equipment. The Navy and Army were facing similar staffing 
pressures as the Air Force a decade ago. Indeed, in 2004, National 
Defence’s Materiel Group developed a concept for operations that 
aimed to provide consistency in rolling out an OWSM approach 

across the Canadian Forces. We could find no evidence that this 
concept was implemented. Neither the Navy nor the Army adopted 

a formal program or directive to require their equipment program 
management divisions to review or change their maintenance and repair 
contracting approaches. Rather, the decision to adopt or not to adopt 

a performance-based approach for a given fleet has remained the 
prerogative of the responsible equipment program management teams. 


§.51 In one instance, an Army management team used a 
contracting approach similar to OWSM to support its several fleets 
of wheeled light armoured vehicles. As explained in the Case Study 
on a new contracting approach, the Army claims to have achieved 
positive results with the performance-based contract it set up for 
these fleets in 2004. The Navy also has some experience with long- 
term maintenance contracts for the maintenance of its minor vessels, 
including the Kingston class of maritime coastal defence vessels. 
However, these contracts were not performance-based, and did 

not include the incentive clauses that characterize OWSM contracts. 


5.52 In our opinion, the lack of concerted action and follow-through 
has resulted in slower and more limited implementation than planned 
of the new contracting approach for existing military equipment and 
related contracts. Further, we believe that implementation has relied 
too much on the initiative of selected individuals. As a consequence, 
National Defence has lost opportunities to derive the potential 
benefits of improved performance, improved accountability, and 
reduced costs. Reducing costs, in particular, might have helped to 
reduce the financial gap between demands for maintenance and repair 


and available funding. 
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Case Study—A new contracting approach has led to improved availability 


of wheeled light armoured vehicles 


The Army owns more than 
1,000 wheeled light armoured 
vehicles of different models, 
including the Bison, the Coyote, 
the Light Armoured Vehicle III, 
and the armoured patrol vehicle. 
All of them entered service 
between 1990 and 2006. 


Prior to 2004, the Army’s 
maintenance and repair approach 
for these fleets was based on the ~ 
traditional model: private firms Light armoured vehicle 
were retained for third- and 

fourth-line services through time 

and materials contracts. This approach proved to be inefficient, creating a significant 
backlog of engineering issues. 


In 2004, the Army applied principles similar to those of the Air Force’s Optimized 
Weapon System Management program and awarded General Dynamics Land Systems— 
Canada (GDLS), a specialized Canadian defence company, a three-year sole-source, 
performance-based contract with two one-year options, for an initial maximum value 

of $392 million. Third- and fourth-line work was then centralized in two separate 
locations, with the Army’s Montreal workshop depot kept as a strategic backup in the 
event that demand temporarily exceeded GDLS’s capacity. In addition to carrying out 
maintenance and repair work, GDLS was responsible for managing suppliers and 
subcontractors, as well as for providing training to National Defence personnel. 


The contract was renewed in 2008 for another five years, with 21 one-year options 
to be granted one year at a time if the company meets specified performance targets. 
The total maximum value of this second contract, over 26 years, is about $3.1 billion. 


According to National Defence, the contract has resulted in significant performance 
improvements. For example, the percentage of on-time delivery for Spare parts has 
increased from an average of 75 percent in 2006 to 94 percent in 2009, and the 
percentage of on-time repair and overhaul projects has increased from an average of 
13 percent in 2006 to 97 percent in 2009. The observed reduction of delays has led 
to increased availability of wheeled light armoured vehicles. 


Source: Prepared based on National Defence documents 


5.53 Recommendation. National Defence should review its 
Optimized Weapon System Management (OWSM) concept for 
operations, formally assess whether OWSM should be pursued for 
other existing fleets, and document its decision for each fleet. For fleets 
where OWSM will be pursued, actions should be taken to ensure 
prompt and complete implementation in accordance with the 
Department's plans and priorities. 


The Department’s response. Agreed. National Defence will review 
the OWSM Concept of Operations and update it as required by 
July 2012. National Defence will then investigate the feasibility of 
implementing OWSM to other existing fleets. 
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In-service support—Activities required to 
sustain the operation of a military fleet over its 
lifetime, including engineering, training, 
inspection, maintenance and repair of 
equipment, and provision of spare parts 
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Demonstrating cost savings is a challenge for the Optimized Weapon System 
Management program 


5.54 One of the objectives of the OWSM program was to reduce the 
costs of maintenance and repair contracts. The program directive fixed 
a target of 15 percent savings on maintenance and repair contract 
costs for selected Air Force fleets. This potentially represented 
hundreds of millions of dollars in savings over many years. We 
examined whether the Air Force had achieved this target through 
OWSM program implementation. 


5.55 In its 2009-10 Departmental Performance Report, National 
Defence reported that its OWSM approach had lowered fleet support 
costs. However, National Defence was unable to provide complete 
information to us to support this assertion, primarily because of the 
lack of reliable baseline cost information. As explained in paragraphs 
5.27-5.29, National Defence has not yet fully implemented an 
integrated asset management information system that would provide it 
with the cost information it needs to do a before and after comparison. 
Other factors make comparisons difficult, including aging equipment, 
changing fleet size, the transfer of new responsibilities to the private 
sector, and the fact that the Department does not generally consider 
the cost of its own personnel’s salaries when accounting for its 
maintenance and repair costs. In our opinion, implementing our 
recommendation in paragraph 5.30 would allow National Defence to 
develop reliable baseline cost information for its military fleets. 


5.56 In 2005, National Defence started to formally develop a 
contracting approach for maintenance and repair of new fleets of 
military aircraft, ships, and land vehicles. The resulting In-Service 
Support Contracting Framework (ISSCF) has been applied as policy 
under the Assistant Deputy Minister (ADM) for Materiel since 
July 2008, and became a departmental directive in August 2010. 


5.57 The ISSCF has been developed with the intention of furthering 
the changes that began under the Optimized Weapon System 
Management (OWSM) program. Under the ISSCE there is only one 
prime contractor per fleet, who is awarded both the acquisition and 
in-service support contracts to create a single point of accountability. 
The original equipment supplier is the prime contractor by default. 
The contracted in-service support period can extend up to 20 years, 
and the contract will include many fixed-price elements. Additional 


responsibilities, such as the ownership and management of spare parts, 
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and the management of all subcontractors, are transferred to the prime 
contractor. Exhibit 5.2 compares the ISSCF characteristics to those 
of the OWSM and traditional approaches. 


5.58 The ISSCF has been formally applied within ADM (Materiel) 
since July 2008, although National Defence had applied some of 

the Framework’s principles in several acquisition projects before then. 
Since 2008, two new military fleets have been acquired and subjected 
to the ISSCF process. Project teams within National Defence are 
currently working on the development of in-service support concepts, 
pursuant to the ISSCE for numerous other planned 

equipment acquisitions. 


5.59 The first of the two recent acquisitions is a new fleet of 

17 CC-130 Model J Hercules transport planes, first contracted 

in 2007, and for which deliveries started in 2010. In this case, a 
contract amendment worth $723 million was awarded in 2009 

to provide for maintenance and repair services until mid-2016. 

The Case Study on the Hercules fleets illustrates how maintenance 
arrangements for the Air Force’s old and new Hercules aircraft have 
evolved over time. 


5.60 The second recent acquisition is a new fleet of 15 Chinook 
helicopters, contracted in 2009, with first delivery expected in 2013. 
The acquisition contract for this fleet includes provisions for 20 years 
of in-service support. Pricing negotiations for these provisions are 
scheduled for completion in 2013; their estimated value is several 
billion dollars. Other acquisition projects are currently under way but, 
overall, experience with ISSCF implementation remains limited at 
this point. 


While In-Service Support Contracting Framework’s risks have been identified, 
mitigation measures are limited 


9.61 By favouring the original equipment supplier as the prime 
contractor for maintenance and repair services, and by combining 
acquisition and support contracts into a single procurement process, 
the ISSCF significantly changes the procurement process for these 
services and introduces new risks, as identified by National Defence. 
We examined whether National Defence designed the ISSCF to 
address key risks related to its maintenance and repair activities, and 
whether it adequately assessed and managed the risks associated with 
the approach. 
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Case Study—tThe maintenance approach for the Hercules fleet has evolved 


over the years to meet changing needs 


CC-130 HERCULES — Models E and H 


National Defence has operated fleets of CC-130 Hercules transport aircraft for over 
50 years. Various Hercules models have been used during that time. In early 2011, 
the fleet included 29 active aircraft of three different models. Models E and H were 
acquired in 1960 and 1996, respectively, and will be retired by 2012 and 2017, 
respectively. 


For decades, National Defence has relied on a combination of in-house capacity and 
contracted services to maintain and repair its fleets of Hercules aircraft. First- and 
second-line work was usually carried out by National Defence staff, while third-line 
work was usually conducted by private sector firms under traditional time and 
materials contracts. National Defence managed the supply chain for spare parts 
and was responsible for engineering modifications and configuration manageinent, 
among other tasks. 


In the early 2000s, faced with declining fleet availability, National Defence decided 
to change its maintenance and repair approach for Models E and H, and made plans 
to transfer third-line work for the airframe, the engines, and the avionics systems to 
Optimized Weapon System Management (OWSM) performance-based contracts. The 
Department has since transitioned to the OWSM approach for airframe maintenance, 
but contracted work for the engines and the avionics systems is still managed under 
more than 20 traditional contracts. 


The OWSM airframe contract was awarded in 2005 to Cascade Aerospace, a 
Canadian company, for five and a half years. Under this contract, worth about 
$423 million, the contractor has incentives to meet performance targets that are 
defined each year in an annual operating plan. Twice a year, a contract performance 
review board assesses the contractor's performance, and decides whether a 
performance bonus or penalties are warranted. In 2010, the contract was renewed 
for an additional five-year term, based on good performance results. 


CC-130 HERCULES — Model J 


In 2007, National Defence contracted Lockheed Martin, an American aerospace 
company, to provide it with 17 new Model J aircraft, which will gradually come into 
service between 2010 and 2012. This new Hercules fleet will be maintained under an In- 
Service Support Contracting Framework (ISSCF) contract. This in-service support contract, 
which has a maximum value of $723 million, was awarded to Lockheed Martin in 
2009, and will extend until mid-2016, with potential optional years beyond that. 


Under this contract, National Defence remains responsible for the first line of 
maintenance and portions of the second line, while the contractor has responsibility 
for the remaining second-line activities, as well as for all third-line maintenance and 
repair work for all systems. The contractor is also responsible for the supply chain, the 
management of all subcontractors, and in-service support for the Department's 
training facility. This arrangement provides National Defence with a single point of 
accountability for aircraft maintenance and repair. 


When fully implemented, this contract structure will enable National Defence to hold 
the contractor accountable for aircraft availability. The contractor will be required to 
provide an annual average of 11.5 planes out of 17 (68 percent) per day. The 
contractor's performance will be measured against this standard, and once a year, this 
performance will be assessed to determine whether any of the penalties defined in the 
contract need to be applied. National Defence personnel will retain responsibilities for 
contract oversight and strategic decisions. 


Source: Prepared based on National Defence documents 
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Directed contract—A federal government 
contract awarded to a preselected contractor 
when the contracting authority can justify 
setting aside the requirement to solicit 
competitive bids. 


5.62 The ISSCF development started in 2005, supported by personnel 
who had experience with the OWSM approach. Essentially, National 
Defence developed the ISSCF as the logical extension of OWSM to 
apply to new acquisitions. We found that rather than conducting a 
new analysis of the risks to maintenance and repair activities, the 
Department accepted that the previously identified pressures and 

risks that gave rise to the OWSM program were still valid: traditional 
time and materials contracts did not promote innovation and high 
performance in industry, lacked sufficient accountability, were resource 
intensive for the Department to manage, and had proven to be 
operationally ineffective, often resulting in poor equipment availability. 


9.63 National Defence acknowledges that the ISSCF is based in part 
on untested concepts related to long-term performance-based 
contracts, and reliance on a single point of accountability for 
maintenance and repair performance and fleet availability. The 
Department identified a number of new risks created by the ISSCF 
itself, recognizing that it could have unforeseen and unintended 
adverse consequences. These risks include 


* loss of work traditionally conducted in Canada if ISSCF contracts 
are awarded to foreign suppliers; 


* total dependency on one supplier for each fleet; 


* reduced financial flexibility and ability to change requirements and 
priorities as needed, because of long-term, fixed-price contracts; 


° loss of skills and expertise required to assess value for money and 
industry proposals, resulting from the transfer of responsibilities to 
the private sector; 


* overpayment for services, especially in a directed contract 
situation; and 


* uncertainty that the required culture change will be successful 
within federal departments faced with having to adapt to the new 
contracting approach. 


9.64 The ISSCF includes some potential measures to mitigate these 
identified risks. While some of these measures are within National 
Defence’s control, others, such as the ISSCF’s call for developing a 
culture change plan for the Government of Canada, depend on the 
authority and collaboration of other federal organizations: And still 
others, such as the training of civilian and military personnel by the 
original equipment supplier, depend on industry. While some measures 
have already been acted upon, others are still at the planning stage, or 
will be implemented only when new acquisition contracts are awarded. 
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5.65 Although we recognize that ISSCF implementation is still in its 
early days, we are concerned that some risks may not be sufficiently 
mitigated by the identified measures. For example, a risk exists that 
long-term contracts could limit the Department’s financial and 
operational flexibility to adapt to changing circumstances in the future. 
Inflexible, fixed-price long-term contracts for specific fleets may reduce 
the amount of funding available for other fleets. We asked the 
Department to provide us with more information on the discussions 
and analysis that were related to the risk of reduced financial flexibility. 
Officials told us that discussions had been held on this topic at senior 
levels, and that the ISSCF had been modified to require that contracts 
be flexible by including predetermined maintenance and repair costs 
for increasing or decreasing levels of equipment usage (for example, 
yearly flying hours). However, no detailed financial analysis was 
prepared to support the risk analysis in this area. Considering the 
potential consequences that a loss of financial flexibility in the 
National Procurement budget could have on the Department's military 
capabilities, we are concerned that a comprehensive financial analysis 
was not undertaken. 


5.66 More importantly, while processes exist within National Defence 
to review in-service support concepts for individual projects, we are 
concerned that there is no departmental forum that provides oversight 
of overall ISSCF implementation, and that reviews risk mitigation over 
the long term. In our opinion, National Defence has not demonstrated 
that it is sufficiently monitoring and mitigating key risks, like the risk of 
losing important maintenance and repair expertise within the 
Department, nor is it providing sufficient assurance that these risks will 
be adequately mitigated. Reducing ISSCF risks will be important, given 
that most of the new major military equipment acquisitions planned 
under the Canada First Defence Strategy for the next two decades will 
be subject to ISSCF requirements. 


5.67 Recommendation. National Defence should regularly review 
the In-Service Support Contracting Framework to update its 
assessment of risks as well as mitigation measures and their status. It 
should identify specific actions and responsibilities for doing this, and 


monitor progress on an ongoing and long-term basis. 


The Department’s response. Agreed. National Defence will examine 
the ISSCF’s risk assessment and mitigation measures with a view 
towards identifying specific actions that might be warranted, and to 


establish an ongoing monitoring system by July 2012. 
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Insufficient resources and oversight threaten implementation of the In-Service 
Support Contracting Framework 


§.68 The ISSCF has significant implications for National Defence 
beyond risk management. For example, implementation requires 
considerably detailed upfront planning by National Defence personnel, 
because they need to make projections over a 20-year period, and to 
consider different scenarios about future fleet usage and maintenance 
and repair needs. We examined whether National Defence had put in 
place actions and oversight mechanisms to ensure successful 
implementation. 


5.69 ‘To support implementation of the ISSCF by individual project 
teams, National Defence staff prepared an action plan in 2008. We 
found that this plan remains unapproved. In addition, because of a 
lack of assigned resources, many of its action items have not been 
undertaken or completed, such as 


* developing a model ISSCF contract, 
¢ drafting a contract performance management framework, 
* preparing a procurement skills development plan, and 


¢ rolling out a communication plan about ISSCE 


During the course of our audit, the central team responsible to support 
ISSCF implementation had only two members, only one of whom was 
assigned full-time. 


5.70 Regarding oversight mechanisms, project teams for new 
acquisitions are required to submit in-service support concept 
proposals for new acquisitions to the Assistant Deputy Minister 
(ADM) for Materiel for approval. Any deviation from the framework 
requirements must also be approved. In practice, the proposals have 
been presented to the Management Group Program Management 


Committee (MGPMC) chaired by the ADM (Materiel). 


5.71 ‘To facilitate its oversight of individual proposals, in March 2009 
the MGPMC directed that a project review committee be established 
for the purpose of independently reviewing in-service support concept 
proposals before they are submitted to the MGPMC. However, this 
did not happen. In its absence, there is no independent review 
process to assess in detail whether in-service support concept proposals 
comply with ISSCF requirements, and to ensure consistency 

in application. 
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5.72 We reviewed all six presentations made to the MGPMC by 
acquisition project teams between 2008 and 2011. We noted 
significant variations related to the level of detail they provided, and 
to the point in the acquisition process at which they were presented. 
In several cases, a support concept proposal was approved by the 
MGPMC, even though the project was still at the conceptual stage and 
neither the specific type of equipment, nor the manufacturer, had yet 
been identified. We found that there is no clear guidance about the 
proper timing for support concepts to be presented to the MGPMC or 
ADM (Materiel), nor about the need to return to MGPMC or ADM 
(Materiel) when project details are more fully known, or in the event 
of significant changes to a procurement project or an in-service 
support concept. 


5.73 While the MGPMC reviews and approves proposals on a project- 
by-project basis, this committee is not dedicated solely to ISSCF 
implementation. As already noted, there is no departmental forum that 
provides oversight of ISSCF implementation, like the Board of Directors 
did for the Optimized Weapon System Management program. 


5.74 Overall, we are concerned that limited central resources and 
oversight threaten the successful implementation of ISSCE We do not 
believe that ISSCF implementation has received the attention and 
resources it needs. Our recommendation on the implementation of 


ISSCF is found at paragraph 5.79. 


Coordination with other federal departments and Canadian industry needs 
to be strengthened 


5.75 As noted in the Introduction to this chapter, the government 
plans to spend substantial amounts of money over 20 years on new 
military fleets and associated maintenance and repair. The Treasury 
Board Contracting Policy requires that contracting for goods and 
services be conducted in a manner that will support long-term 
industrial and regional development. Public Works and Government 
Services Canada (PWGSC), the department responsible for 
government contracts, and Industry Canada, responsible for industrial 
regional benefits, both have roles to play in the procurement of new 
military equipment. Thus, ISSCF implementation requires 
collaboration with these federal departments and others, as necessary, 


to ensure consistent and effective application. 


§.76 National Defence is expected to have the technical expertise to 
draft statement of requirements or statement of work that will be used 


in contractual agreements. The management of a contract is typically 
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done jointly between National Defence and PWGSC to best benefit 
from the expertise of both departments. PWGSC, by contrast, is 
expected to ensure that its staff has the specialized skills required to 
draft complex performance-based contractual agreements, monitor 
them over long periods, and effectively challenge claims from 
contractors, especially in directed contracts, where the risk of paying 
higher prices is greater. In doing such work, PWGSC must work closely 
with National Defence. However, National Defence took no action to 
ensure that PWGSC would be able to support ISSCF implementation 
as intended. We also noted that, in the internal implementation action 
plan, resources have not been assigned to any of the identified actions 
that relate to working with other federal departments. 


5.77 Beyond federal departments, the ISSCF also has potential 
implications for the Canadian defence industry. Current trends suggest 
that more and more responsibility for providing maintenance and 
repair services is being transferred to the private sector. Thus, it is 
important that National Defence both understand and support 
industry capacity. In the course of developing the ISSCF National 
Defence informed industry of the direction it intended to take, and 
made some changes to the framework in reaction to concerns 
expressed at the time. Notwithstanding, industry associations have 
more recently expressed concern about designating the original 
equipment supplier as the default prime contractor for maintenance 
and repair. Because few manufacturers of military aircraft, ships, and 
land vehicles exist in Canada, industry is concerned that opportunities 
for Canadian defence companies will diminish, and that most of the 
value-added work, like engineering design, will be done outside 
Canada. National Defence believes that robust application of the 
government’s Industrial Regional Benefits policy will address these 
concerns. The ISSCF does not include a plan or mechanism to 
monitor its impact on the Canadian defence industry as 
implementation proceeds, or to address issues as they arise. 


9.78 National Defence will need to strengthen internal processes and 
better coordinate its efforts with other federal departments and the 
Canadian defence industry to ensure that maintenance and repair 
contracts for all new acquisitions provide good value for Canadian 
taxpayers, good results for the Canadian Forces, and good business 
opportunities for the Canadian defence industry. 


9.79 Recommendation. National Defence should review and revise 
the In-Service Support Contracting Framework (ISSCF) governance 
structure to ensure adequate and timely departméntal oversight and 
control. The Department should adequately resource the specific 


30 


Chapter 5 


Report of the Auditor General of Canada—Fall 2011 


MAINTAINING AND REPAIRING MILITARY EQUIPMENT—NATIONAL DEFENCE 


actions it has identified that need to be taken to ensure ISSCF’s 
successful implementation, including the need to coordinate its efforts 
with other federal departments and the Canadian defence industry. 


The Department’s response. Agreed. By July. 2012, National Defence 
will review its governance structure to ensure adequate and timely 
visibility and control of: (a) fleet support concepts; and (b) ISSCF 
institutionalization. National Defence has already identified 

specific actions that need to be undertaken to ensure successful 
institutionalization, including the need to coordinate its efforts with 
other federal departments and Canadian industry; resourcing of those 
actions is in progress. 


Conclusion 


5.80 We witnessed dedicated military and civilian personnel in the 
field working diligently to achieve assigned objectives and missions. 
Our audit determined that National Defence planned and managed 
the maintenance and repair of military equipment to meet operational 
priorities in the short term. However, its ability to meet training and 
operational requirements over the long term is at risk, because of long- 
standing deficiencies in management information systems, the lack of 
sufficient cost and performance information, and ongoing weaknesses 
in the implementation and oversight of its contracting approaches for 
maintenance and repair services. 


5.81 Regarding maintenance and repair funding allocations, we 
determined that National Defence has an effective forum in place to 
allocate financial resources to defined priorities. However, National 
Defence has limited ways of assessing the long-term impacts of its 
annual National Procurement (NP) allocation decisions and lacks 
important information on total costs and performance of maintenance 
and repair activities. Without this information, National Defence has a 
limited basis on which to determine whether it is putting sufficient 
funding into maintenance and repair activities each year to optimize 


its support of military equipment over the long term. 


5.82 Regarding contracting approaches, we found that National 
Defence did establish maintenance and repair contracting frameworks 
for its military vehicles, ships, and aircraft in response to risks and 
pressures it was facing at the time. However, implementation of the 
Optimized Weapon System Management approach for existing fleets 
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has been slower and more limited than intended. Consequently, 
opportunities for improvements have been missed and benefits have 
been postponed. 


5.83 Implementation of the In-Service Support Contracting 
Framework (ISSCF) for new military equipment is still at an early stage. 
Many actions needed to ensure successful implementation by individual 
project teams are yet to be approved, and central resources required for 
implementation have been limited. Although risks associated with the 
ISSCF have been identified, many mitigation measures have yet to be 
put in place, and there is no suitable forum to oversee implementation. 
Considering the long-term implications and risks for National Defence, 
other federal departments, and the Canadian defence industry, we do 
not believe that ISSCF implementation has received the attention and 
resources it needs to ensure success. 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by the Canadian Institute of Chartered Accountants. While the Office adopts these 
standards as the minimum requirement for our audits, we also draw upon the standards and practices of 
other disciplines. 


Objectives 


The overall objective of the Audit was to determine whether National Defence planned and managed the 
maintenance and repair of military aircraft, ships, and land vehicles to meet operational and training 
requirements. 


The two sub-objectives were as follows: 


- To determine whether National Defence appropriately allocated and monitored financial resources to 
maintain and repair its military aircraft, ships, and land vehicles to meet operational and training 
requirements. 


- To determine whether National Defence appropriately established and implemented maintenance and 
repair contracting approaches for its military aircraft, ships, and land vehicles. 


In this audit, “appropriately” means making decisions based on a clearly communicated rationale that 
takes into account risks and departmental directives. 


Scope and approach 


Our audit focused on the allocation and management of financial resources for maintenance and repair 
activities of military equipment, as well as on contracting approaches used by National Defence for 
maintenance and repair services. 


The audit examined how National Defence allocated its financial resources to its priorities for the 
maintenance and repair of military equipment. Specifically, we examined whether National Defence 

had systems and practices to provide managers with the information necessary to make decisions about 
allocating financial resources for maintenance and repair purposes, as well as how this information was used. 


The audit also examined how National Defence designed and implemented contracting approaches for the 
maintenance and repair of military equipment. Specifically, we examined whether National Defence's new 
contracting approaches were based on a strategic analysis of the risks to which the Department's 

equipment assets and maintenance program activities were exposed, and whether the risks created by the 
new approaches were adequately assessed and managed. Finally, we examined whether National Defence 


had appropriately implemented its new contracting approaches. 


The audit examined documents contained in National Defence files, as well as maintenance and repair 
contracts for military fleets managed under the Optimized Weapon System Management and In-Service 
Support Contracting Framework approaches. We did not audit the records of private sector firms. 


Accordingly, our conclusions do not pertain to private sector practices. 
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During the course of the audit, we conducted over 100 interviews with National Defence officials and 
Canadian Forces members located in the Department’s headquarters in Ottawa or on Canadian Forces 
bases (CFBs) across Canada. We visited maintenance and repair installations used by the Army 

(CFB Montreal and CFB Valcartier), the Air Force (CFB Shearwater and CFB Trenton), and the Navy 


(CFB Esquimalt and CFB Halifax). 


We also met with representatives of private sector companies that were awarded long-term maintenance 
and repair contracts. for Canadian Forces aircraft, ships, or land vehicles. As part of this work, we 
interviewed representatives for associations of Canadian defence and aerospace companies. We did not 


audit private sector entities. 


In addition, we had discussions on maintenance and repair contracting approaches with our counterparts in 
national audit offices in the United States, the United Kingdom, and the Netherlands. We also had similar 
discussions with representatives of the defence departments of the United Kingdom and the Netherlands. 


The audit did not examine recruitment and retention practices related to maintenance personnel, nor did 


it examine the management of infrastructure used for maintenance purposes. We did not audit the 


portions of military equipment acquisition contracts that were not related to maintenance and repair. 


Criteria 


Criteria 


National Defence allocates its budgeted resources toward its 
priorities for the maintenance and repair of military equipment. 


To determine whether National Defence appropriately allocated and monitored fi rancial 
its military aircraft, ships, and land vehicles to meet operational and t 


¢ Guide to Management of Materiel, Treasury Board of Canada 
Secretariat 


National Defence has systems and practices that provide 
information necessary to make decisions about allocating financial 
resources for maintenance and repair of military equipment. 


¢ Policy on Management of Materiel, Treasury Board, 2006 


The Army, Navy, and Air Force manage equipment readiness 
planning and monitoring to meet operational and ve Bees: 


¢ Guide to Management of Materiel, Treasury Board of 
Us eeu a 


National Defence develops maintenance approaches for the 
maintenance and repair of military equipment that are based on 
a strategic analysis of the risks to which the Department’s 
equipment assets and maintenance program activities and 
interests are exposed. 


for its military aircraft, ships, and land vehicles, \ 


° Paticy'o on Aenea of Materiel, Treasury Board, 2006 
e Risk Management Policy, Treasury Board, 1994 


National Defence appropriately implements applicable 
approaches to maintenance and repair. 


¢ The Way Ahead (Optimized Weapon System Management 
(OWSM) directive and guidance), National Defence 


In-Service Support Contracting Framework for Canadian Forces 
Platforms During the Initial Acquisition Stage, 
National Defence, 2008, 2009 


¢ Defence Administrative orders and directives 3022-0 
and 3022-1, National Defence 


Management reviewed and accepted the suitability of the criteria used in the audit. 
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Period covered by the audit 


Our examination of the management and allocation of the maintenance and repair funds covered the 
planning cycles for fiscal years 2009-10 and 2010-11. Our examination of contracting approaches covered 


the decade from 2001 to 2011. 


Audit work for this chapter was substantially completed on 30 April 2011. 


Audit team 


Assistant Auditor General: Jerome Berthelette 
Principal: John Reed 
Directors: Casey Thomas and Dusan Duvnjak 


Pierre Fréchette 
Eric Provencher 


Mathieu Tremblay 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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Appendix List of recommendations 


The following is a list of recommendations found in Chapter 5. The number in front of the 
recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses 
indicate the paragraphs where the topic is discussed. 


Allocation and monitoring of financial resources 


5.30. National Defence should ensure Agreed. Project Management Office (PMO) MASIS is rolling 


that it develops the ability to produce out Systems, Applications, and Products (SAP) in support of 
overall and fleet-specific total cost maintenance of weapon systems and equipment. The SAP 
information for its maintenance and system, called the Defence Resource Management Information 
repair activities. These costs should System (DRMIS) is currently in place for over 90% of the Army 
include, at a minimum, expenses and Navy, and approximately 10% of the Air Force. There are 
related to personnel, contracted plans to implement DRMIS in the remainder of the Canadian 
services, spare parts, maintenance Forces and this system has the capacity to meet the information 
equipment, and infrastructure. requirements identified in this recommendation. This is planned 
(5.14—-5.29) for completion by December 2013. 


The MASIS Phase V on-going project has integrated spares 
inventory with procurement, financial and maintenance 
engineering data under a single platform. This integration paves 
the way to equipment, fleet and overall cost reporting. As the 
implementation of MASIS progresses, personnel, spare parts and 
procurement costs are being captured. At this time, 
infrastructure cost is not captured in MASIS, but will be in the 
future as the Department continues to advance its Enterprise 
Resource Planning strategy and its SAP implementation. 


In parallel, the Repair and Overhaul business process involving 
the management of the repair line will be assessed against best _ 
industry practices to determine the improvements required to 
optimize this part of the supply chain. This will take place prior 
to MASIS Phase V blueprinting for repairs and overhaul 
scheduled in the fall of 2011. 


The Maritime equipment program management division is 
actively moving towards a class-focused program that will 
improve the ability to capture total maintenance and repair costs 
for each class of naval platform. 


Finally, the work being conducted in developing costing tools 
and guidelines in response to the Auditor General’s audit report 
on the Acquisition of Military Helicopters will also assist in 
meeting this recommendation. 
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5.37 National Defence should develop 


and implement a capacity to provide 
information on the performance and 
impacts of maintenance and repair 
activities, their effectiveness, and their 
efficiency within each of the Army, 
Navy, and Air Force, using common 
performance measures where possible. 


(5.31-5.36) 


Agreed. As noted in the previous response, PMO MASIS is 
rolling out the Defence Resource Management Information 
System (DRMIS) in support of maintenance of weapon systems 
and equipment, which will have the capacity to provide the 
information outlined in this recommendation. This 
implementation is planned for completion by December 2013. 


Business Intelligence specific queries made available through 
MASIS Phase V enable some searches on equipment and 
environment-specific performance and measures. Further 
development is required to introduce advanced queries and tools 
to enhance this Department's ability to perform forward 
planning so that overall effectiveness and efficiency are 
improved. The Department is also pursuing work on the 
development of performance measures in the area of inventory 
management. 


Contracting approaches for existing equipment 


§.53 National Defence should review 
its Optimized Weapon System 
Management (OWSM) concept for 
operations, formally assess whether 
OWSM should be pursued for other 
existing fleets, and document its 
decision for each fleet. For fleets where 
OWSM will be pursued, actions should 
be taken to ensure prompt and 
complete implementation in 
accordance with the Department’s 
plans and priorities. (5.38-5.52) 


Agreed. National Defence will review the OWSM Concept of 
Operations and update it as required by July 2012. National 
Defence will then investigate the feasibility of implementing 


OWSM to other existing fleets. 


nD 


Contracting approaches for new equipment 


5.67. National Defence should 
regularly review the In-Service Support 
Contracting Framework to update its 
assessment of risks as well as mitigation 
measures and their status. It should 
identify specific actions and 
responsibilities for doing this, and 
monitor progress on an ongoing and 


long-term basis. (5.56—5.66) 
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assessment and mitigation measures with a view towards 
identifying specific actions that might be warranted, and to 
establish an ongoing monitoring system by July 2012. 
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5.79 National Defence should review Agreed. By July 2012, National Defence will review its 


and revise the In-Service Support governance structure to ensure adequate and timely visibility 
Contracting Framework (ISSCF) and control of: (a) fleet support concepts; and (b) ISSCF 
governance structure to ensure institutionalization. National Defence has already identified 
adequate and timely departmental specific actions that need to be undertaken to ensure successful 


oversight and control. The Department __ institutionalization, including the need to coordinate its efforts 
should adequately resource the specific with other federal departments and Canadian industry; 

actions it has identified that need tobe —_ resourcing of those actions is in progress. 

taken to ensure ISSCF’s successful 

implementation, including the need to 

coordinate its efforts with other federal 

departments and the Canadian defence 


industry. (5.68—5.78) 
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to be laid before the House in accordance with the provisions of subsection 7(3) of the Auditor General Act. 


John Wiersema, FCA 
Interim Auditor General of Canada 


OTTAWA, 22 November 2011 


er kien jaw ty) el 
“a me erabetimmemerec Til donee Cok 
‘3 


~~ x 


> 
i 
7) = 
eA = SY sth 
we en: ~ weg renee 
errs m= 4 
f 
6 
: a ~ 
ms ‘ 
r= Gaye et | 
‘ 
: : oa 
§- ‘ 7 
> 2 a p z Ps 
1 7 _ i 
i, a Pane t 
= Se =< - (hs O89 
=) u 4 7 
we . 
‘ - - a y= ot 
- a 2 
Pe u as : aad 
wn 4 a. Va 7 - an 7 


Table of Contents 


Matters of Special Importance—2011 


Main Points—Chapters 1 to 5 


Chapter 1 Canada’s Economic Action Plan 

Chapter 2 Issuing Visas 

Chapter 3 Payments to Producers—Agriculture and Agri-Food Canada 
Chapter 4 Regulating Pharmaceutical Drugs—Health Canada 

Chapter 5 Maintaining and Repairing Military Equipment—National Defence 


Appendices 


A. Auditor General Act 
B. Reports of the Standing Committee on Public Accounts to the House of Commons, 2010-11 


C. Report on the audit of the President of the Treasury Board’s Annual Report to Parliament 
on the Tabling of Crown Corporations’ Reports 2011 


D. Costs of Crown corporation audits conducted by the Office of the Auditor General of Canada 


Report of the Auditor General of Canada—Fall 2011 


39 
40 


. or. 
-_ i= 
i i) 
a 
i) 
7 
! lentil oo =e! 
om . : 7 a eh ad “y 
: ee ; (ot) 7 bee Tee ae meena recta o« 
- a 1 eae etic rie ey. 8 
, . A =f. (rea AT aU Pn ep = Sigal PY? 
\ i | | : ? - = 
‘ 4? 
; on, Oy ste) - 
_ 7 ,40¥ i ea of. ay 7 pei 
V ALAS. ARS % eS (inci \ pit Gael? @ re 
i 1) (here * reaviator rap 9) Ty Saye eM | 
he eS a 
n g . 


Matters of Special 
Importance—2011 


— » 
45 enEM 
vi 


anireryad 


- 


John Wiersema, FCA 
Interim Auditor General of Canada 


Report of the Auditor General of Canada 


Fa 


201) 


“Vii 


Matters of Special Importance—201 1 


I am pleased to present this report to Parliament, covering a range of 
matters that indicate the diversity of activities in which the federal 
government engages on behalf of Canadians: 


* implementing Canada’s Economic Action Plan; 

* issuing visas for entry into Canada; 

¢ making income support payments to agricultural producers; 

* regulating pharmaceutical drugs; and 

* maintaining and repairing military equipment. 
My appointment as Interim Auditor General of Canada was recent and 
my tenure will be brief. Nonetheless, my 33 years in the Office provide 
a vantage point from which | have observed that government programs 


and initiatives must have certain fundamental elements to be 
successful. They include 


* clarity of purpose, 


committed and sustained leadership, 


* predictable and stable funding, and 


sufficient and appropriate information for management. 


This observation is supported by many reports we have published over 
my years with the Office. For this report, we have drawn examples 
from some of the chapters issued during 2011. 


Clarity of purpose. Successful government programs require a long- 
term vision of what the program is intended to achieve and a strategic 
plan to guide activities. Where programs or initiatives involve more 
than one organization—inside or outside the federal government— 


roles and responsibilities need to be clearly defined. 


In our first audit of Canada’s Economic Action Plan, we found that 
decision makers shared a clear sense of the initiative’s purpose— 

to stimulate the Canadian economy by funding projects that were ready 
to begin and could be completed within two years. To accomplish this, 
central agencies and departments worked together to achieve timely 
implementation; billions of dollars were quickly dispersed while 
considerable attention was paid to safeguarding against risk and 
ensuring that eligibility criteria were met. (2010 Fall Report, Chapter 1) 
Our second audit found that the federal organizations we examined 


knew they had to manage the risks involved in funding close 
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to 6,000 projects within a fixed two-year period. They monitored the 
risks, tracked progress and spending, and took corrective action as 
needed. (2011 Fall Report, Chapter 1) 


Our June Status Report chapter on programs for First Nations noted 
that conditions remain significantly poorer on reserves than in other 
communities, and are even worsening. This is due in part to a long- 
standing lack of clarity about responsibility for on-reserve services such 
as drinking water, housing, and education. First Nations, the federal 
government, and other stakeholders need to find new ways of working 
together to achieve better results for First Nations communities. (June 
2011 Status Report, Chapter 4) 


Another Status Report chapter, on National Police Services, 
highlighted a continuing lack of clarity in the RCMP about which 
police services it should be providing to provinces and municipalities, 
at what level, and who should pay for them. These matters remain 
unresolved despite the fact that it has been providing these services 
since 1966. The RCMP has continued to struggle with the rising costs 
of providing national police services, at the expense of its federal 
policing responsibilities in areas such as organized crime and drug 
enforcement. (June 2011 Status Report, Chapter 5) 


Committed and sustained leadership. The complexity of many 
government programs can require years of sustained leadership to 
achieve desired results. This is often difficult to accomplish given 

the high rates of turnover in senior positions. While the internal audit 
function is a relatively small part of government activity, our Status 
Report discussed the progress the government has made in improving 
it. This was achieved largely because of committed leadership. Across 
the government, we found stronger senior management support for the 
role internal audit can play. The Office of the Comptroller General 
has provided leadership, guidance, and tools to the internal audit 
community. In addition, independent departmental audit committees 
have been established whose collective skills and experience enable 
them to provide deputy heads of departments with objective advice 
and recommendations. (June 2011 Status Report, Chapter 3) 


However, we have often seen government programs or initiatives 
where stronger leadership is needed to resolve long-standing problems. 
The chapter in this report on issuing visas provides an example. 


The chapter notes some problems that we have identified in our audits 
of Citizenship and Immigration Canada (CIC) since 2000. For example, 
tools and guidance available to visa officers are not kept up to date; 
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screening for danger to public health is still primarily based on the same 
diseases 


syphilis and tuberculosis—as 50 years ago, though today . 
some 56 diseases are reportable in Canada; and many risk indicators, 
key to identifying potentially inadmissible foreign nationals, have not 
been reviewed or updated for years. Committed leadership could have 
ensured long ago that these issues were addressed. At the same time, 
CIC and the Canada Border Services Agency, who have shared 
responsibility since 2004 for security aspects of admissibility, are still 
dealing with problems in their relationship and clarifying how they will 
work together. (2011 Fall Report, Chapter 2) 


In October 2011, the Commissioner of the Environment and 
Sustainable Development noted that the Government of Canada 
has committed to addressing climate change by reducing its national 
greenhouse gas (GHG) emissions in various plans and agreements 
since 1992. However, national GHG emissions in 2008 were 

24 percent higher than in 1990. Further, in its first climate change 
plan, the government was aiming to reduce emissions by 282 million 
tonnes; by the 2010 plan, it was aiming to reduce them by only 

28 million tonnes. The government needs to establish a target it is 
committed to achieving, as well as the means of doing so. 


(October 2011 CESD Report, Chapter 1) 


Our Spring 2011 chapter on the Reserve Force Pension Plan found 
that no one senior official was made responsible for the plan. This 
resulted in a division of responsibilities and a lack of coordinated 
leadership. Parliament authorized the creation of this plan in 1999. 
Eight years later, in 2007, National Defence introduced the plan but 
was not prepared to operate it. After three years, only 400 of the 
9,000 pension buyback applications by reservists had been processed; 
by 2014 about 3,900 files could still be in the backlog. Many reservists 
will wait seven years or longer to find out what their pension will be 
and how much it will cost them. (2011 Spring Report, Chapter 3) 


Predictable and stable funding. The business of government is long- 
term in nature, and many government programs require many years to 
achieve the desired results. In contrast, the government's funding cycle 
for non-statutory programs is short-term in its focus. In addition, many 
ongoing programs depend on time-limited funding that has to be 
renewed periodically to allow the program to continue. 


In our Status Report on programs for First Nations on reserves, we 
noted the difficulties caused by the lack of an appropriate funding 
mechanism for services. Funds are appropriated annually and provided 


through contribution agreements, most of which must be renewed 
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each year. The funds may not be available until several months into 
the period to be funded; one reason is that new agreements cannot 

be finalized until departments have reviewed documentation and 
confirmed that funds from the previous period were used appropriately. 
Consequently, First Nations must often reallocate funds from 
elsewhere to continue meeting community service requirements. 

This creates uncertainty and instability for those who rely on the 
services. (June O11 Status Report, Chapter 4) 


In the same report, we found that the RCMP has been reallocating 
funds from other programs and activities to cover the rising costs of 
providing national police services. To balance its resources without 
additional funding, the RCMP will have to either reduce these services 
or sustain them by permanently redirecting funding from other RCMP 
programs. (June 2011 Status Report, Chapter 5) 


Similarly, in 2006, we reported that demands on Health Canada’s 
Medical Devices Program were increasing with the number of devices 
in the marketplace and the complexity of new technologies and 
applications. Yet the program’s funding remained constant, even 
decreasing for core activities, and it was becoming more difficult 

for program managers to fully meet regulatory responsibilities for 
protection of Canadians’ health and safety. This year, our Status 
Report noted that to help resolve the shortfall, Health Canada has 
increased funding to the program and updated user fees. (June 2011 
Status Report, Chapter 6) 


Predictable, stable funding contributes to the ability to manage a 
program strategically and for the long term, but only if decision makers 
understand the full long-term costs. Before decisions are made to go 
forward with an initiative that commits the government to a future 
course of action involving years or even decades, it is important that 
decision makers consider all of the costs, not just the start-up costs. 


National Defence has pointed to a significant gap between the demand 
for maintenance and repair services and the funds made available. 

It also says its long-term investment plan for new equipment has 
likely allocated insufficient funds for equipment life-cycle costs. 

The Department has found that the impacts of postponing 
maintenance and repair work include the need for more expensive 
corrective repairs, and reduced availability and life expectancy of 
military equipment. (2011 Fall Report, Chapter 5) 


Sufficient and appropriate information for management. Costs are 
just one type of information that decision makers need. Our audits 
have found that essential pieces of information needed to manage 
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the complex and diverse business of government—information on not 
only costs but also objectives, results, and service levels—are often 
missing or inadequate. 


Federal departments make significant investments in systems that 
generate huge amounts of information. The government’s ability to 
carry out policies and programs that will serve the needs of Canadians 
relies on having the right kind of information and then using it to 
manage well. 


Organizations need good financial and other types of information to 
monitor the delivery of programs and services, exercise stewardship 
over resources for which they are responsible, support decisions, 
manage risks, and report on the financial and operating results. 


The significant gaps in information needed to understand and respond 
to the changing state of our environment has been a recurring theme 
in reports by the Commissioner of the Environment and Sustainable 
Development. In his October 2011 Report, for example, he notes that 
insufficient environmental information makes it difficult to understand 
the combined impacts of oil sands projects in the lower Athabasca 
region and on ecosystems farther afield. Without knowing the 
combined effects, decisions about oil sands projects have been based 
on incomplete, poor, or non-existent environmental information. 


(October 2011 CESD Report, Chapter 2) 


The chapter on issuing visas notes that CIC expects its visa officers to 
make the best decisions on the admissibility of a visa applicant that 
they can make in the time and with the information available. 
Unfortunately, management at CIC and CBSA do not know whether 
the security and health information provided to visa officers is sound, 
because there is very little checking on its quality. CBSA has not 
requested feedback from CIC on the usefulness of the information 
provided to visa officers, and there is no process to find out how they 
use the information. About 45 percent of the visa officers we surveyed 
indicated that one challenge in determining the admissibility of an 
applicant is the lack of relevant information from security partners. 


(2011 Fall Report, Chapter 2) 


Our Status Report noted that the RCMP’s ability to negotiate 
agreements with provinces and municipalities on the provision of 
national police services is limited by its lack of adequate information 
on the cost of providing each service. Accurate cost information is 
essential to manage its services and resources efficiently. june 2011 


Status Report, Chapter 5) 
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Our second chapter on the Economic Action Plan notes that job 
creation was a key objective of one program we looked at, and 
contribution agreements included specific performance indicators 
related to job creation. However, job information was collected in a 
variety of ways and cannot be relied on to assess the program’s success 
in meeting this key objective. In addition, fragmented reporting on the 
performance of Economic Action Plan programs in departmental 
performance reports has made it difficult to get a picture-of the 
initiative’s overall results. (2011 Fall Report, Chapter 1) 


Over the years, we have issued few positive reports on information for 
management. Instead, we have found that poor information is a 
widespread, chronic problem in the federal government. We have seen 
many initiatives that require managers to feed information to the 
centre, whether to central agencies or to departmental corporate 
services—the reporting burden that exists in government. In contrast, 
managers are not systematically collecting and using the information 
they need to manage their programs, and they are not held 
accountable for this. 


Conclusion 


The government will need to make significant investments to upgrade 
information systems and aging infrastructure, among others. This will 
require careful long-range planning, predictable and stable funding, and 
good information as well as good use of that information. In addition, 
committed and sustained leadership and clarity are needed as to the 
roles and responsibilities of public officials charged with managing the 
complex and challenging issues that the government is facing. 


These matters are important and timely in light of the government’s 
current strategic review. Carrying out the review will be challenging for 
programs that lack one or more of these essential elements. I encourage 
the government to ensure that programs continuing after completion 
of the review have these elements in place. I also encourage the 
government to ensure that managers have the necessary tools, people, 
and information systems to achieve government priorities. While they 
must be left to manage, it will be important that managers be held to 
account for results achieved. 
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Chapter 1 


What we examined 
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Canada’s Economic Action Plan 


Main Points 


In January 2009, the Government of Canada launched its Economic 
Action Plan to stimulate the economy in response to the global 
economic downturn. This stimulus plan represented about $47 billion 
in federal spending and an additional $14 billion funded by provinces 
and territories. Its purpose was to create jobs, build infrastructure, 
accelerate housing construction, stimulate spending by Canadians, 
and support businesses and communities. Budget 2009 also contained 
measures to add stability to the financial sector, which sought to 
improve access to financing for consumers and business by providing 
up to $200 billion in credit. 


Our first audit of the Economic Action Plan, reported in October 2010, 
examined program design and delivery mechanisms put in place by 
selected federal departments and agencies to implement the Economic 
Action Plan. 


Our second audit of the Economic Action Plan is the subject of this 
chapter. The audit looked at three programs with a total dollar value 

of $7 billion. The $4 billion Infrastructure Stimulus Fund targeted 
provincial, territorial, and municipal construction-ready projects to build 
or rehabilitate infrastructure. The $2 billion Knowledge Infrastructure 
Program targeted post-secondary institutions across Canada for new 
construction, deferred maintenance, repair, and expansion of projects to 
improve the quality of research and development and deliver advanced 
knowledge and skills training. The $1 billion Community Adjustment 
Fund was established to create or maintain jobs and support businesses 


in communities hit hard by the economic downturn. 


Our audit included two departments and five regional development 
agencies that distributed funds under the programs. We examined 
whether they had monitored and reported on program spending 


and results. 
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Why it’s important 


What we found 


During our audit, the government extended the completion deadline 
for many projects from 31 March 2011 to 31 October 2011. This 
chapter is a report on selected programs up to 31 March 2011, the date 
when our audit work was substantially completed. It is not a report on 
the final results of the Economic Action Plan. 


Given the short time frame planned for the Economic Action Plan, 

as well as the large amounts of public money involved, it was important 
that the government mitigate risks through appropriate monitoring of 
programs at every stage, timely tracking and reporting of costs, 
adherence to its own Policy on Transfer Payments, and reliable 
reporting of results. 


A key risk for the government was that its measures would fail to 
quickly stimulate the Canadian economy. That was the reason for 
giving priority to projects that were “construction ready” and requiring 
that projects be completed by 31 March 2011, to support the 
government’s assertion that stimulus measures would be targeted, 
timely, and temporary. 


* For the three Economic Action Plan programs we audited, the 
federal government monitored the progress and spending of projects, 
permitting it to take corrective action in a number of cases. Progress 
on many projects was slower than initially expected. Departments 
and agencies delivering the three programs that we audited reported 
that 4,070 out of 5,845 projects (70 percent) were completed by 
the 31 March 2011 deadline. Following the federal government’s 
announcement of a deadline extension, almost one third of projects 
in the two largest programs we examined were granted an extension 
to 31 October 2011. The decision to extend the time frames of these 
programs was supported by an analysis conducted by the Privy 
Council Office and the Department of Finance Canada. 


Spending figures provided by the departments and agencies as 
of 31 March 2011 indicate that the three programs had largely 
achieved the Economic Action Plan objective to spend federal 
resources within a two-year time frame. However, total federal 
spending for all three programs will not be known until projects 
submit final claims and close-out reports to the federal entities. 


Although a key objective of the Community Adjustment Fund was 
to create and maintain jobs in communities hit hard by the economic 
downturn, the design of the program did not allow for performance 
measurement and reporting against this key objective. 
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+ Infrastructure Canada, Industry Canada, and the regional 
development agencies reported performance information on their 
Economic Action Plan programs in various places throughout their 
departmental performance reports. In our view, this fragmented 
presentation makes it difficult for parliamentarians and Canadians to 
obtain an overall picture of results achieved against planned 
performance expectations and public resources spent. 


The entities have responded. The entities agree with our 
recommendations. Their detailed responses follow each 
recommendation throughout the chapter. 
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What we examined 


Issuing Visas 


Main Points 


People from other countries—foreign nationals—who want to enter 
Canada as permanent residents must obtain a Canadian visa. Foreign 
nationals who want to enter Canada on a temporary basis must also 
apply for a visa, unless they are from a visa-exempt country. To obtain a 
visa, foreign nationals must meet all requirements for the category 
under which they are applying and must be deemed to be admissible. 
In 2010, 1.36 million visas (including 317,000 permanent resident 
visas) were processed at Canadian missions in foreign countries. 


Admissibility of foreign nationals into Canada falls under the 
Immigration and Refugee Protection Act. The Act defines various 
situations where a foreign national would be inadmissible—for 
example, if the individual presents a risk to the health, safety, or 
security of Canadians. Administering the various provisions of the Act 
is a shared responsibility between Citizenship and Immigration Canada 


(CIC) and the Canada Border Services Agency (CBSA). 


Before issuing a visa, CIC officials must determine that the applicant is 
admissible to Canada. They are supported in making this 
determination by the CBSA, which—with the help of the Canadian 
Security Intelligence Service (CSIS) and the Royal Canadian Mounted 
Police (RCMP)—coordinates and provides intelligence information 
related to the applicants. 


We examined whether Citizenship and Immigration Canada and the 
Canada Border Services Agency have managed the risks associated 
with determining admissibility before issuing a visa, in line with the 
objective of the Act to protect the health, safety, and security of 
Canadians. 


Audit work for this chapter was substantially completed on 
29 April 2011. 
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Why it’s important = Global events in the last decade have changed the nature of threats to 
Canadian society. Diseases prevalent in other countries that can be 
transmitted rapidly worldwide, incidents of terrorism, and organized 
crime around the world have shown the importance of identifying 
individuals who present a risk and preventing their entry into Canada. 


Identifying visa applicants who are inadmissible to Canada is a highly 
complex process that relies heavily on the judgment and experience of 
CIC’s visa officers in missions overseas and on the information made 
available to them. Visa officers are expected to make the best decisions 
they can with that information in the time they have available. It is 
critical that visa officers receive from their security and medical 
partners timely and reliable information on applicants. 


What we found ~— + Citizenship and Immigration Canada and the Canada Border 
Services Agency have taken some measures to address long-standing 
weaknesses in the process of determining whether visa applicants are 
admissible to Canada. However, deficiencies still exist in the measures 
used to identify foreign nationals who may be inadmissible for health, 
safety, or security concerns. CIC and the CBSA lack the necessary 
tools and information to provide assurance that risks related to the 
admissibility determination process are properly managed. 


* Some of the tools and risk indicators that visa officers use to identify 
inadmissible persons, and to know when to seek advice from security 
partners, are not kept up to date, nor are they always available. 
Furthermore, many CBSA analysts who provide security advice to visa 
officers have not received the necessary formal training to do so. 
Documentation to support the advice sent to visa officers offered little 
insight into how the analysts made their assessments, and in many 
cases not all the checks that should have been done were completed. 


* CIC lacks guidance on the use of two key criteria used in medical 
screening—danger to public health and danger to public safety— 
although it has undertaken some work to explain what they mean. 
Medical screening to determine danger to public health has focused 
mainly on the same two diseases for the past 50 years—syphilis and 
tuberculosis. Although today 56 diseases require national 
surveillance in Canada, CIC has not reviewed whether foreign 
nationals should also be subject to mandatory testing for some of 


these diseases. 


T 
| 
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* CIC and the CBSA do not have systematic quality assurance 
practices or performance measures in place to know how well they 
are identifying individuals who are inadmissible because of health, 
safety, or security concerns. Most quality assurance practices that do 
exist focus on supporting decisions to refuse a visa. Because those 
decisions represent a very small percentage of applications each year, 
this means that the quality of decisions on the vast majority of 
applications is not reviewed. 


The entities have responded. The entities agree with all of our 
recommendations. Their detailed responses follow the 
recommendations throughout the chapter. 
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What we examined 


Report of the Auditor General of Canada—Fall 2011 


Payments to Producers— 
Agriculture and Agri-Food Canada 


Main Points 


Agriculture and Agri-Food Canada (AAFC) has a mandate to provide 
information, research and technology, and policies and programs to 
achieve an environmentally sustainable, innovative, and competitive 
agriculture sector. The Department’s work includes supporting 
productivity and trade, stabilizing farm incomes, and conducting 
research. The federal government and the provinces and territories 
share responsibility for stabilizing farm incomes. Since 2008, they have 
done so through a joint federal/provincial/territorial agreement called 
the Growing Forward Framework Agreement. 


The Department’s programs have played an important role in 
supporting producers’ incomes when market income has dropped. 
Our audit looked at two programs under Growing Forward that, like 
the Canadian Agricultural Income Stabilization (CAIS) program they 
replaced, are aimed at protecting agricultural producers from drops in 
income. Agrilnvest is built around savings accounts, with producers’ 
deposits matched by government contributions; AgriStability is a far 
more complex program designed to protect against larger drops in 
income. Costs of the two programs total $1 billion annually, funded 
60 percent by the federal government and 40 percent by provinces. 
About 88 percent of payments under the AgriStability program are 
now delivered by the provinces. 


We also looked at the $284 million Tobacco Transition Program, aimed 
at replacing the quota system, helping tobacco producers move out of 
the tobacco industry, and improving the viability of those who remain. 


We looked at how the programs were developed, implemented, and 
administered. In particular, we looked at the quality of the risk 
assessment process and the process for continuous improvement. We also 
looked at the design and monitoring of funding and program delivery 
agreements between the Department and the provinces and territories or 
delivery organizations. We did not audit the provincial agencies or 


third-party organizations that participated in the delivery of programs. 


Audit work for this chapter was substantially completed 
on 2 May 2011. 
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Why it’s important 


What we found 


The Canadian agriculture and agri-food industry is vital to Canada’s 
economic success and its food supply. It encompasses several 
industries, including primary agriculture, input suppliers, food and 
beverage processing and distribution, and wholesale and retail food 
industries. According to the Department, this industry accounted for 
8.2 percent of total gross domestic product in 2009. 


The agricultural sector faces several challenges, including increasing 
international competition, rapid technological improvements, 
increased importance of environmental and health concerns, increasing 
input costs, rapidly evolving consumer preferences, changes in foreign 
exchange, and more volatility due to weather changes and disease. 


- To improve the design of its producer income support programs, 
Agriculture and Agri-Food Canada carried out industry 
consultation, a strategic review, producer surveys, and focus groups. 
Working with provinces and territories, the Department has made 
progress on some design issues, but long-standing concerns with 
AgriStability remain—clarity of program objectives, timely access to 
program funding, and program complexity that affects producers’ 
ability to reasonably predict payment amounts. 


The Department has made progress in addressing program 
administration issues raised in our 2007 audit—for example, 
payment accuracy and management of underpayments to producers. 
However, it has not systematically followed up on causes of delays 
in payments to producers in order to accelerate payments. Despite 
improvement, the Department has not yet met its AgriStability 
processing time targets, and producers can wait up to two years after 
an income loss to receive a payment. In addition, the Department 
has not collected information on processing times for Agrilnvest. 


Although federal/provincial/territorial accountabilities for 
performance reporting were not specified in the Growing Forward 
Framework Agreement, a performance measurement framework has 
since been agreed to by all governments. When fully implemented, 
overall national performance against service standards will be 
reported to Parliament and the public. With regard to assessing 
provincial capacity for AgriStability administration and transferring 
it to the provinces of British Columbia and Saskatchewan in 2010, 
the Department followed a sound process. 


* The Department had to develop the Tobacco Transition Program 
within a short time frame and did not first conduct a thorough risk 
analysis. The agreement implementing the program did not provide 
clear terms and conditions to ensure that recipients would not enter 
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into business arrangements that would undermine the intent of the 
program. In addition, a number of times the Department changed its 
interpretation of what was and was not allowed under the Agreement, 
resulting in confusion for producers. As a result, the Department was 
successful in controlling some, but not all, business arrangements that 
it believed would undermine the intent of the program. 


The Department has responded. The Department agrees with all of 
our recommendations. Its responses follow each recommendation 
throughout the chapter. 
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Chapter 4 


What we examined 


Why it’s important 


Regulating Pharmaceutical Drugs— 


Health Canada 


Main Points 


Pharmaceutical drugs are mostly synthetic products made from 
chemicals. They are meant to improve the health and well-being of 
patients by helping to prevent and treat disease, reduce pain and 
suffering, and extend and save lives. Some higher-risk drugs, such as 
those used to treat diseases, require a prescription from a physician. 
Other lower-risk drugs, such as cough syrup and antacids, are sold 
without a prescription and are readily available to the public. 


Health Canada, through the Food and Drugs Act, regulates the safety, 
efficacy, and quality of all pharmaceutical drugs for use by humans in 
Canada before and after the products enter the Canadian marketplace. 
The Department does this through a combination of scientific review, 
monitoring, compliance, and enforcement activities. It aims to ensure 
that the public has timely access to safe and effective pharmaceutical 
drugs and that those who need to know of safety concerns are informed. 


We examined how Health Canada regulates clinical trials of new 
pharmaceutical drugs and reviews submissions seeking approval of new 
drugs for sale in Canada or of changes to drugs already on the market. 
We also examined how the Department monitors product safety and 
ensures that potential safety concerns are communicated to health 
care professionals and the public. In addition, we looked at how Health 
Canada enforces industry compliance with regulatory requirements 
governing the testing, production, and sale of drugs. We did not 
examine the soundness of the Department’s regulatory decisions or the 
safety or efficacy of drugs. 


The period under audit for this chapter was 1 January 2009 to 
31 December 2010. Audit work for this chapter was substantially 
completed on 31 May 2011. 


There are about 13,000 prescription and non-prescription drugs on 
the Canadian market. Pharmaceutical drugs play an important role 
in Canada’s health care system and economy. In 2008, the Canadian 
retail market for prescription and over-the-counter drugs was valued 
at about $28 billion, with prescription drug purchases accounting for 
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almost 84 percent of total retail drug expenditures. According to IMS 
Brogan, a well-recognized provider of data to Health Canada and the 
pharmaceutical industry, about 505 million prescriptions were 
dispensed by Canadian retail pharmacies in 2010. 


With an aging population, the role of pharmaceutical drugs is expected 
to grow as researchers come up with new drug therapies to replace 
earlier treatments or provide new options where no treatment existed 
before. Canadians who purchase and consume pharmaceuticals 
authorized for sale in Canada rely on the government and industry to 
monitor the safety of these products. Health Canada has a 
responsibility to help protect the public against undue health and 
safety risks from the use of pharmaceutical drugs. 


« The Department does not take timely action in its regulatory 
activities, with the exception of its review of two types of drug 
submissions. In particular, the Department is slow to assess potential 
safety issues. It can take more than two years to complete an 
assessment of potential safety issues and to provide Canadians with 
new safety information. 


The Department received 4,400 drug submissions in 2009 and 2010. 
It has put in place processes and procedures to ensure that its drug 
reviews are consistent and high quality. However, it has not assessed 
whether these processes and procedures have been consistently 
interpreted and applied across its four review bureaus. 


Health Canada does not disclose information on drug submissions 
that it has rejected or information on the status of the drugs it has 


approved with conditions. In addition, the Department has not 
acted on its long-standing commitment to disclose more information 
about clinical trials it has authorized. This increases the risk that 
Canadians may be unaware of new treatment options or may 
unknowingly participate in an unauthorized trial. 


» Health Canada’s conflict-of-interest guidelines and Code of Conduct 
are consistent with government policy on conflict of interest. 
However, unlike another major regulator of pharmaceutical drugs and 
some federal departments that have developed conflict-of-interest 
requirements for specific work assignments, the Department has not 
determined what measures are necessary for its review activities. 


The Department has responded. The Department agrees with 
all of our recommendations. Its detailed responses follow the 


recommendations throughout the chapter. 
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What we examined 


Why it’s important 


Maintaining and Repairing Military 
Equipment—National Defence 


Main Points 


National Defence and the Canadian Forces own, operate, and are 
responsible for maintaining and repairing military aircraft, ships, and 
land vehicles costing more than $30 billion. In 2009-10, National 
Defence spent more than $2 billion to maintain and repair its military 
equipment. This included expenses for routine inspections, preventive 
maintenance, corrective repairs, spare parts supply, periodic repair and 
overhaul, engineering changes, and other related tasks. 


Thousands of personnel are engaged in maintenance and repair 
activities, which include everything from turning wrenches on bases or 
while deployed on missions to engineering, logistics and spare parts 
management, training, contracting, and administrative support. 


We examined how National Defence allocates and manages financial 
resources for the maintenance and repair of its military equipment. 
We also examined its approaches to contracting for maintenance and 
repair services. 


Audit work for this chapter was substantially completed 
on 30 April 2011. 


To undertake training and to meet assigned missions safely and 
successfully, military equipment must be kept in good working 
condition and be ready for action on short notice. How National 
Defence allocates the funds available—and the reliability of the 
information it uses to support short- and long-term decisions—is 
critical to the ability and readiness of the Canadian Forces to meet 
their assigned missions. 


Military spending on maintenance and repair also makes a significant 
contribution to the Canadian economy every year. Many Canadian 
companies depend on it for a portion of their business. According to 
the 2008 Canada First Defence Strategy, the government plans to 
spend $60 billion on new military equipment over 20 years and 

$140 billion for spare parts, maintenance, and training. 
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Over the last decade, National Defence has made sweeping changes in 
its approach to contracting for maintenance and repair of both existing 
and new equipment. The changes transfer much of the responsibility 
to the private sector, with significant implications for National Defence 
and Canada’s defence industry. The new contracting approach for 
existing equipment was intended to reduce the Department’s contract 
management activities and costs by bundling hundreds of short-term 
maintenance contracts into a few longer-term contracts. The approach 
for new equipment goes even further, awarding both the acquisition 
and the long-term maintenance and repair contracts to the original 
equipment manufacturer or supplier. The Department has identified 
significant risks in this approach, including limited flexibility if 
requirements change over the life of the equipment, dwindling 
maintenance and repair skills and expertise in the Canadian Forces, 
and total dependence on one supplier for each fleet. 


* Overall, National Defence has planned and managed the 
maintenance and repair of military equipment to meet operational 
priorities in the short term. The annual process of allocating 
available funds provides an effective forum to discuss priorities, 
with wide participation of those responsible for maintaining and 
repairing military equipment and those who need it for operations 
and training. 


* National Defence’s ability to meet training and operational 
requirements over the long term is at risk due to weaknesses 
in implementation and oversight of its contracting approaches 
for maintenance and repair, deficient management information 
systems, and the lack of sufficient cost and performance information. 


- The Department has not taken the actions or provided the central 
resources and oversight required to support the implementation of its 
new contracting approaches successfully. The lack of concerted action 
and follow-through on the new contracting approach for existing 
military equipment has resulted in slower and more limited 
implementation than planned. As a consequence, National Defence 
has lost opportunities to derive the potential benefits of improved 
performance, improved accountability, and reduced costs. In addition, 
National Defence is not adequately monitoring and mitigating the 
significant risks created by its approach for new equipment. 


- There are long-standing deficiencies in information management 
systems used to support decision-making for maintenance and 
repair activities, first raised by us in a 2001 audit. As a result, 
National Defence lacks complete, reliable, and integrated 
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information on the total actual costs of maintenance and repair, 
because some of the costs—salaries and infrastructure, for 
example—are not captured in its asset management information 
systems. The absence of this information impedes its ability to make 
informed decisions about the life-cycle management of its fleets or to 
determine whether it is putting enough funds each year into 
maintenance and repair. In 2001, National Defence expected to fully 
implement an integrated asset management system by 2004. The 
Department now expects a new system to be introduced on all 
Canadian Forces bases by mid-2012. Fully implementing this new 
system will likely take many years. 


- There is a significant gap between the demand for maintenance and 


repair services and the funds made available. In addition, National 
Defence has indicated it is likely that its long-term investment plan 
for new equipment has allocated insufficient funds for equipment 
life-cycle costs. Although National Defence knows that postponing 
maintenance and repair tasks creates future risks—such as reduced 
availability of equipment, more laborious and expensive repairs, and 
reduced life expectancy of military equipment—the Department 
does not regularly monitor these impacts. Consequently, it does not 
know the specific long-term impacts of the funding gap on operations 
and training activities. 


The Department has responded. The Department agrees with 
all of our recommendations. Its detailed responses follow the 
recommendations throughout the chapter. 
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Appendix A Auditor General Act 


Short title 


Definitions 


“appropriate Minister” 


“Auditor General” 


“category I 
department” 


“Commissioner” 
“Crown corporation” 
“department” 
“funding agreement” 


“recipient” 


“registrar” 


An Act respecting the office of the Auditor General of Canada 
and sustainable development monitoring and reporting 


Short Title 

1. This Act may be cited as the Auditor General Act. 
Interpretation 

Zz In this Act, 


“appropriate Minister” has the meaning assigned by section 2 of the Financial 
Administration Act; 


“Auditor General” means the Auditor General of Canada appointed pursuant to 
subsection 3(1); 


“category I department” means 


(a) any department named in schedule I to the Financial 
Administration Act, 


(b) any department in respect of which a direction has been made 
under subsection 11(3) of the Federal Sustainable Development Act; 
and 


(c) any agency set out in the schedule to the Federal Sustainable 
Development Act. 


“Commissioner” means the Commissioner of the Environment and Sustainable 
Development appointed under subsection 15.1(1); 


“Crown corporation” has the meaning assigned to that expression by section 83 of 
the Financial Administration Act; 


“department” has the meaning assigned to that term by section 2 of the Financial 


Administration Act; 


“funding agreement” has the meaning given to that expression by subsection 


42(4) of the Financial Administration Act; 


“recipient” has the meaning given to that expression by subsection 42(4) of the 


Financial Administration Act; 


“registrar” means the Bank of Canada and a registrar appointed under Part IV of 


the Financial Administration Act; 
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“sustainable 
development” 


Control 


Control 


Appointment 


‘Tenure 


Re-appointment 


Interim appointment 


“sustainable development” means development that meets the needs of the 
present without compromising the ability of future generations to meet their own 
needs; 


2.1 (1) For the purpose of paragraph (d) of the definition “recipient” in 
subsection 42 (4) of the Financial Administration Act, a municipality or 
government controls a corporation with share capital if 


(a) shares of the corporation to which are attached more than fifty per 
cent of the votes that may be cast to elect directors of the 
corporation are held, otherwise than by way of security only, by, on 
behalf of or in trust for that municipality or government; and 


(b) the votes attached to those shares are sufficient, if exercised, to 
elect a majority of the directors of the corporation. 


(2) For the purpose of paragraph (d) of the definition “recipient” in 
subsection 42 (4) of the Financial Administration Act, a corporation without share 
capital is controlled by a municipality or government if it is able to appoint the 
majority of the directors of the corporation, whether or not it does so. 


Auditor General of Canada 


3. (1) The Governor in Council shall, by commission under the Great Seal, 
appoint an Auditor General of Canada after consultation with the leader of every 
recognized party in the Senate and House of Commons and approval of the 
appointment by resolution of the Senate and House of Commons. 


(1.1) The Auditor General holds office during good behaviour for a term of 
10 years but may be removed for cause by the Governor in Council on address of 
the Senate and House of Commons. 


(2) [Repealed, 2011, c. 15, s. 17] 


(3) Once having served as the Auditor General, a person is not eligible for 
re-appointment to that office. 


(4) In the event of the absence or incapacity of the Auditor General or if 
that office is vacant, the Governor in Council may appoint any qualified auditor 
to hold that office in the interim for a term not exceeding six months, and that 
person shall, while holding office, be paid the salary or other remuneration and 
expenses that may be fixed by the Governor in Council. 
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Salary 4. (1) The Auditor General shall be paid a salary equal to the salary of a 
puisne judge of the Supreme Court of Canada. 


Pension benefits (2) The provisions of the Public Service Superannuation Act, other than 
those relating to tenure of office, apply to the Auditor General except that a 
person appointed as Auditor General from outside the public service may, by 
notice in writing given to the President of the Treasury Board not more than sixty 
days after the date of his appointment as Auditor General, elect to participate in 
the pension plan provided for in the Diplomatic Service (Special) Superannuation 
Act in which case the provisions of that Act, other than those relating to tenure 
of office, apply to him and the provisions of the Public Service Superannuation Act 
do not apply to him. 


Powers and Duties 


Examination 5. | The Auditor General is the auditor of the accounts of Canada, including 
those relating to the Consolidated Revenue Fund and as such shall make such 
examinations and inquiries as he considers necessary to enable him to report as 
required by this Act. 


Idem 6. The Auditor General shall examine the several financial statements 
required by section 64 of the Financial Administration Act to be included in the 
Public Accounts, and any other statement that the President of the Treasury 
Board or the Minister of Finance may present for audit and shall express his 
opinion as to whether they present fairly information in accordance with stated 
accounting policies of the federal government and on a basis consistent with that 
of the preceding year together with any reservations he may have. 


Annual and additional 7. (1) The Auditor General shall report annually to the House of Commons 

reports to the House and may make, in addition to any special report made under subsection 8(1) or 

wi gains: 19(2) and the Commissioner’s report under subsection 23(2), not more than 
three additional reports in any year to the House of Commons 


(a) on the work of his office; and, 


(b) on whether, in carrying on the work of his office, he received all the 
information and explanations he required. 


Report of the Auditor General of Canada— Fall 2011 Appendices | 27 


APPENDICES 


Idem (2) Each report of the Auditor General under subsection (1) shall call 
attention to anything that he considers to be of significance and of a nature that 
should be brought to the attention of the House of Commons, including any 
cases in which he has observed that 


(a) accounts have not been faithfully and properly maintained or 
public money has not been fully accounted for or paid, where so 
required by law, into the Consolidated Revenue Fund; 


(b) essential records have not been maintained or the rules and 
procedures applied have been insufficient to safeguard and control 
public property, to secure an effective check on the assessment, 
collection and proper allocation of the revenue and to ensure that 
expenditures have been made only as authorized; 


(c) money has been expended other than for purposes for which it was 
appropriated by Parliament; 


(d) money has been expended without due regard to economy or 
efficiency; 
(e) satisfactory procedures have not been established to measure and 


report the effectiveness of programs, where such procedures could 
appropriately and reasonably be implemented; or 


(f) money has been expended without due regard to the 
environmental effects of those expenditures in the context of 
sustainable development. 


Submission of annual (3) Each annual report by the Auditor General to the House of Commons 
report to Speaker and 
tabling in the House 
of Commons 


shall be submitted to the Speaker of the House of Commons on or before 
December 31 in the year to which the report relates and the Speaker of the 
House of Commons shall lay each such report before the House of Commons 
forthwith after receiving it or, if that House is not then sitting, on any of the first 
fifteen days on which that House is sitting after the Speaker receives it. 


Notice of additional (4) Where the Auditor General proposes to make an additional report 
reports to Speaker and ynder subsection (1), the Auditor General shall send written notice to the 


tabling in the House x 
ee pee ae Speaker of the House of Commons of the subject-matter of the proposed report. 


Submission of (5) Each additional report of the Auditor General to the House of 
additional reports (0 = Commons made under subsection (1) shall be submitted to the House of 
Speaker and tabling ie: ous 

ALT a, Commons on the expiration of thirty days after the notice is sent pursuant to 
Commons subsection (4) or any longer period that is specified in the notice and the Speaker 


of the House of Commons shall lay each such report before the House of 
Commons forthwith after receiving it or, if that House is not then sitting, on any 
of the first fifteen days on which that House is sitting after the Speaker receives it. 
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Inquiry and report 7.1 (1) The Auditor General may, with respect to a recipient under any funding 
agreement, inquire into whether 


(a) the recipient has failed to fulfil its obligations under any funding 
agreement; 


(b) money the recipient has received under any funding agreement has 
been used without due regard to economy and efficiency; 


(c) the recipient has failed to establish satisfactory procedures to 
measure and report on the effectiveness of its activities in relation 
to the objectives for which it received funding under any funding 
agreement; 


(d) the recipient has failed to faithfully and properly maintain accounts 
and essential records in relation to any amount it has received 
under any funding agreement; or 


(e) money the recipient has received under any funding agreement has 
been expended without due regard to the environmental effects of 
those expenditures in the context of sustainable development. 


Report (2) The Auditor General may set out his or her conclusions in respect of an 
inquiry into any matter referred to in subsection (1) in the annual report, or in 
any of the three additional reports, referred to in subsection 7(1). The Auditor 
General may also set out in that report anything emerging from the inquiry that 
he or she considers to be of significance and of a nature that should be brought to 
the attention of the House of Commons. 


Special report tothe 8. = (1) The Auditor General may make a special report to the House of 

House of Commons = Commons on any matter of pressing importance or urgency that, in the opinion 
of the Auditor General, should not be deferred until the presentation of the next 
report under subsection 7(1). 


Submission of reports (2) Each special report of the Auditor General to the House of Commons 

ssi Speaker and tabling ade under subsection (1) or 19(2) shall be submitted to the Speaker of the 

oh rte - House of Commons and shall be laid before the House of Commons by the 
Speaker of the House of Commons forthwith after receipt thereof by him, or if 
that House is not then sitting, on the first day next thereafter that the House of 


Commons is sitting. 
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Idem 


Improper retention of 
public money 


Inquiry and report 


Advisory powers 


Access to information 


Stationing of officers 
in departments 


9. The Auditor General shall 


(a) make such examination of the accounts and records of each 
registrar as he deems necessary, and such other examinations of a 
registrar’s transactions as the Minister of Finance may require, and 


(b) when and to the extent required by the Minister of Finance, 
participate in the destruction of any redeemed or cancelled 
securities or unissued reserves of securities authorized to be 
destroyed under the Financial Administration Act, 


and he may, by arrangement with a registrar, maintain custody and control, 
jointly with that registrar, of cancelled and unissued securities. 


10. Whenever it appears to the Auditor General that any public money has 
been improperly retained by any person, he shall forthwith report the 
circumstances of the case to the President of the Treasury Board. 


11. The Auditor General may, if in his opinion such an assignment does not 
interfere with his primary responsibilities, whenever the Governor in Council so 
requests, inquire into and report on any matter relating to the financial affairs of 
Canada or to public property or inquire into and report on any person or 
organization that has received financial aid from the Government of Canada or in 
respect of which financial aid from the Government of Canada is sought. 


12. The Auditor General may advise appropriate officers and employees in the 
federal public administration of matters discovered in his examinations and, in 
particular, may draw any such matter to the attention of officers and employees 
engaged in the conduct of the business of the Treasury Board. 


Access to Information 


13. (1) Except as provided by any other Act of Parliament that expressly refers 
to this subsection, the Auditor General is entitled to free access at all convenient 
times to information that relates to the fulfilment of his or her responsibilities and 
he or she is also entitled to require and receive from members of the federal 
public administration any information, reports and explanations that he or she 
considers necessary for that purpose. 


(2) In order to carry out his duties more effectively, the Auditor General 
may station in any department any person employed in his office, and the 
department shall provide the necessary office accommodation for any person so 
stationed. 
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(3) The Auditor General shall require every person employed in his office 
who is to examine the accounts of a department or of a Crown corporation 
pursuant to this Act to comply with any security requirements applicable to, and 
to take any oath of secrecy required to be taken by, persons employed in that 
department or Crown corporation. 


(4) The Auditor General may examine any person on oath on any matter 
pertaining to any account subject to audit by him and for the purposes of any 
such examination the Auditor General may exercise all the powers of a 
commissioner under Part I of the Inquiries Act. 


14. (1) Notwithstanding subsections (2) and (3), in order to fulfil his 
responsibilities as the auditor of the accounts of Canada, the Auditor General 
may rely on the report of the duly appointed auditor of a Crown corporation or of 
any subsidiary of a Crown corporation. 


(2) The Auditor General may request a Crown corporation to obtain and 
furnish him with such information and explanations from its present or former 
directors, officers, employees, agents and auditors or those of any of its 
subsidiaries as are, in his opinion, necessary to enable him to fulfil his 
responsibilities as the auditor of the accounts of Canada. 


(3) If, in the opinion of the Auditor General, a Crown corporation, in 
response to a request made under subsection (2), fails to provide any or sufficient 
information or explanations, he may so advise the Governor in Council, who may 
thereupon direct the officers of the corporation to furnish the Auditor General 
with such information and explanations and to give him access to those records, 
documents, books, accounts and vouchers of the corporation or any of its 
subsidiaries access to which is, in the opinion of the Auditor General, necessary 
for him to fulfil his responsibilities as the auditor of the accounts of Canada. 


Staff of the Auditor General 


15. (1) The officers and employees that are necessary to enable the Auditor 
General to perform his or her duties are to be appointed in accordance with the 
Public Service Employment Act and, subject to subsections (2) to (5), the 
provisions of that Act apply to those officers and employees. 


(2) The Auditor General may exercise the powers and perform the 
functions of the employer and deputy head under the Public Service Employment 
Act within the meaning of those terms in subsection 2(1) of that Act. 


———— ——— a re Appendices 31 


APPENDICES 


Public Service 
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Sub- delegation 


(3) The Auditor General may, in the manner and subject to the terms and 
conditions that the Public Service Commission directs, exercise the powers and 
perform the functions of that Commission under the Public Service Employment 
Act, other than its powers and functions in relation to the hearing of allegations 
by a candidate under sections 118 and 119 of that Act and its power to make 
regulations. 


(4) The Auditor General may authorize any person employed in his or her 
office to exercise and perform, in any manner and subject to any terms and 
conditions that he or she directs, any of his or her powers and functions under 
subsections (2) and (3). 


(5) Any person authorized under subsection (4) may, subject to and in 
accordance with the authorization, authorize one or more persons under that 
person’s jurisdiction to exercise any power or perform any function to which the 
authorization relates. 


15.1 (1) The Auditor General shall, in accordance with the Public Service 
Employment Act, appoint a senior officer to be called the Commissioner of the 
Environment and Sustainable Development who shall report directly to the 
Auditor General. 


(2) The Commissioner shall assist the Auditor General in performing the 
duties of the Auditor General set out in this Act that relate to the environment 
and sustainable development. 


16. The Auditor General is authorized, in respect of persons appointed in his or 
her office, to exercise the powers and perform the functions of the Treasury Board 
that relate to human resources management within the meaning of paragraph 
7(1)(e) and section 11.1 of the Financial Administration Act, as well as those of 
deputy heads under subsection 12(2) of that Act, as that subsection reads 
without regard to any terms and conditions that the Governor in Council may 
direct, including the determination of terms and conditions of employment and 
the responsibility for employer and employee relations. 


16.1 (1) The Auditor General may authorize any person employed in his or her 
office to exercise and perform, in any manner and subject to any terms and 
conditions that he or she directs, any of his or her powers and functions in 
relation to human resources management. 


(2) Any person authorized under subsection (1) may, subject to and in 
accordance with the authorization, authorize one or more persons under that 
person’s jurisdiction to exercise any power or perform any function to which the 
authorization relates. 
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16.2 Subject to any other Act of Parliament or regulations made under any Act 
of Parliament, but without the approval of the Treasury Board, the Auditor 
General may, within the total dollar limitations established for his or her office in 
appropriation Acts, contract for professional services. 


17. Classification standards may be prepared for persons employed in the office 
of the Auditor General to conform with the classifications that the Auditor 
General recognizes for the purposes of that office. 


18. The Auditor General may designate a senior member of his staff to sign on 
his behalf any opinion that he is required to give and any report, other than his 
annual report on the financial statements of Canada made pursuant to section 64 
of the Financial Administration Act and his reports to the House of Commons 
under this Act, and any member so signing an opinion or report shall indicate 
beneath his signature his position in the office of the Auditor General and the 
fact that he is signing on behalf of the Auditor General. 


Immunities 


18.1 The Auditor General, or any person acting on behalf or under the direction 
of the Auditor General, is not a competent or compellable witness — in respect 
of any matter coming to the knowledge of the Auditor General or that person as a 
result of performing audit powers, duties or functions under this or any other Act 
of Parliament during an examination or inquiry — in any proceedings other than 
a prosecution for an offence under section 131 of the Criminal Code (perjury) in 
respect of a statement made under this Act. 


18.2 (1) No criminal or civil proceedings lie against the Auditor General, or 
against any person acting on behalf or under the direction of the Auditor 
General, for anything done, reported or said in good faith in the course of the 
performance or purported performance of audit powers, duties or functions under 
this or any other Act of Parliament. 


(2) For the purposes of any law relating to defamation, 


(a) anything said, any information supplied or any document or thing 
produced in good faith by or on behalf of the Auditor General, in 
the course of the performance or purported performance of audit 
powers, duties or functions under this or any other Act of 


Parliament, is privileged; and 


(b) any report made in good faith by the Auditor General in the course 
of the performance or purported performance of audit powers, 
duties or functions under this or any other Act of Parliament, and 
any fair and accurate account of the report made in good faith in a 
newspaper or any other periodical publication or in a broadcast, is 


privileged. 
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Purpose 


Estimates 


19. (1) The Auditor General shall annually prepare an estimate of the sums 
that will be required to be provided by Parliament for the payment of the salaries, 
allowances and expenses of his office during the next ensuing fiscal year. 


(2) The Auditor General may make a special report to the House of 
Commons in the event that amounts provided for his office in the estimates 
submitted to Parliament are, in his opinion, inadequate to enable him to fulfil the 
responsibilities of his office. 


20. The provisions of the Financial Administration Act with respect to the 
division of appropriations into allotments do not apply in respect of 
appropriations for the office of the Auditor General. 


Audit of the Office of the Auditor General 


21. (1) A qualified auditor nominated by the Treasury Board shall examine the 
receipts and disbursements of the office of the Auditor General and shall report 
annually the outcome of his examinations to the House of Commons. 


(2) Each report referred to in subsection (1) shall be submitted to the 
President of the Treasury Board on or before the 31st day of December in the year 
to which the report relates and the President of the Treasury Board shall lay each 
such report before the House of Commons within fifteen days after receipt 
thereof by him or, if that House is not then sitting, on any of the first fifteen days 
next thereafter that the House of Commons is sitting. 


Sustainable Development 


21.1 In addition to carrying out the functions referred to in subsection 23 (3), 
the purpose of the Commissioner is to provide sustainable development 
monitoring and reporting on the progress of category I departments towards 
sustainable development, which is a continually evolving concept based on the 
integration of social, economic and environmental concerns, and which may be 
achieved by, among other things, 


(a) the integration of the environment and the economy; 
) protecting the health of Canadians; 

) protecting ecosystems; 

(d) meeting international obligations; 

(e) promoting equity; 


(f) an integrated approach to planning and making decisions that 
takes into account the environmental and natural resource costs of 
different economic options and the economic costs of different 
environmental and natural resource options; 
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(g) preventing pollution; and 


(h) respect for nature and the needs of future generations. 


22. (1) Where the Auditor General receives a petition in writing from a 
resident of Canada about an environmental matter in the context of sustainable 
development that is the responsibility of a category I department, the Auditor 
General shall make a record of the petition and forward the petition within 
fifteen days after the day on which it is received to the appropriate Minister for 
the department. 


(2) Within fifteen days after the day on which the Minister receives the 
petition from the Auditor General, the Minister shall send to the person who 
made the petition an acknowledgement of receipt of the petition and shall send a 
copy of the acknowledgement to the Auditor General. 


(3) The Minister shall consider the petition and send to the person who 
made it a reply that responds to it, and shall send a copy of the reply to the 
Auditor General, within 


(a) one hundred and twenty days after the day on which the Minister 
receives the petition from the Auditor General; or 


(b) any longer time, where the Minister personally, within those one 
hundred and twenty days, notifies the person who made the 
petition that it is not possible to reply within those one hundred 
and twenty days and sends a copy of that notification to the 
Auditor General. 


(4) Where the petition is from more than one person, it is sufficient for the 
Minister to send the acknowledgement and reply, and the notification, if any, to 
one or more of the petitioners rather than to all of them. 


23. (1) The Commissioner shall make any examinations and inquiries that the 


Commissioner considers necessary in order to monitor 


(a) the extent to which category I departments have contributed to 
meeting the targets set out in the Federal Sustainable Development 
Strategy and have met the objectives, and implemented the plans, 
set out in their own sustainable development strategies laid before 
the Houses of Parliament under section 11 of the Federal 
Sustainable Development Act; and 


(b) the replies by Ministers required by subsection 22(3). 
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(2) The Commissioner shall, on behalf of the Auditor General, report 
annually to Parliament concerning anything that the Commissioner considers 
should be brought to the attention of Parliament in relation to environmental 
and other aspects of sustainable development, including 


(a) the extent to which category I departments have contributed to 
meeting the targets set out in the Federal Sustainable Development 
Strategy and have met the objectives, and implemented the plans, set 
out in their own sustainable development strategies laid before the 
Houses of Parliament under section 11 of the Federal Sustainable 
Development Act; 


(b) the number of petitions recorded as required by subsection 22(1), 
the subject-matter of the petitions and their status; and 


(c) the exercising of the authority of the Governor in Council under 
subsections 11(3) and (4) of the Federal Sustainable Development Act. 


(3) The Commissioner shall examine the report required under 
subsection 7(2) of the Federal Sustainable Development Act in order to assess the 
fairness of the information contained in the report with respect to the progress of 
the federal government in implementing the Federal Sustainable Development 
Strategy and meeting its targets. 


(4) The results of any assessment conducted under subsection (3) shall be 
included in the report referred to in subsection (2) or in the annual report, or in 
any of the three additional reports, referred to in subsection 7(1). 


(5) The report required by subsection (2) shall be submitted to the 
Speakers of the Senate and the House of Commons and the Speakers shall lay it 
before their respective Houses on any of the next 15 days on which that House is 
sitting after the Speaker receives the report. 
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Appendix B Reports of the Standing Committee on Public Accounts to the House of Commons, 2010-11 


The following reports have been tabled since our October 2010 Report went to print. They are available 
on the website of Canada’s Parliament (www.parl.gc.ca). 


40th Parliament, 3rd Session 


Report 15—Selected Departmental Performance Reports for 2008—2009—Department of Industry, 
Department of Transport (Adopted by the Committee on 10 June 2010; presented to the House 
on 20 September 2010) 


Report 16—Chapter 2, Risks of Toxic Substances, of the Fall 2009 Report of the Commissioner of the 
Environment and Sustainable Development (Adopted by the Committee on 17 June 2010; presented to 
the House on 20 September 2010) 


Report 17—Chapter 1, Evaluating the Effectiveness of Programs, of the Fall 2009 Report of the Auditor 
General of Canada (Adopted by the Committee on 17 June 2010; presented to the House 
on 20 September 2010) 


Report 18—Chapter 8, Strengthening Aid Effectiveness—Canadian International Development Agency, 
of the Fall 2009 Report of the Auditor General of Canada (Adopted by the Committee on 17 June 2010; 
presented to the House on 20 September 2010) 


Report 19—Chapter 5, Acquiring Military Vehicles for Use in Afghanistan, of the Fall 2009 Report of the 
Auditor General of Canada (Adopted by the Committee on 7 October 2010; presented to the House 
on 25 October 2010) : 


Report 20—International Peer Review of the Office of the Auditor General of Canada (Adopted by the 
Committee on 4 November 2010; presented to the House on 24 November 2010) 


Report 21—Main Estimates 2010-2011: Part III—2010-2011 Report on Plans and Priorities and 2008— 
2009 Departmental Performance Report of the Office of the Auditor General of Canada (Adopted by the 
Committee on 25 November 2010; presented to the House on 8 December 2010) 


Report 22—Chapter 1, Canada’s Economic Action Plan, of the Fall 2010 Report of the Auditor General 
of Canada (Adopted by the Committee on 25 November 2010; presented to the House 
on 8 December 2010) 


Report 23—Chapter 4, Electronic Health Records, of the Fall 2009 Report of the Auditor General of 
Canada, and Electronic Health Records in Canada—An Overview of Federal and Provincial Audit 
Reports, of the Spring 2010 Report of the Auditor General of Canada (Adopted by the Committee 
on 4 November 2010; presented to the House on 8 December 2010) 


eee eT ; é ape rue Mee 
Report 24—Chapter 4, Sustaining Development in the Northwest Territories, of the Spring 2010 Report 
of the Auditor General of Canada (Adopted by the Committee on 2 December 2010; presented to the 


House on 2 February 2011) 
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Report 25—Chapter 1, Aging Information Technology Systems, of the Spring 2010 Report of the Auditor 
General of Canada (Adopted by the Committee on 2 December 2010; presented to the House 
on 2 February 2011) 


Report 26—Chapter 2, Modernizing Human Resource Management, of the Spring 2010 Report of the 
Auditor General of Canada (Adopted by the Committee on 9 December 2010; presented to the House 
on 2 February 2011) 


Report 27—Chapter 3, Service Delivery, of the Fall 2010 Report of the Auditor General of Canada 
(Adopted by the Committee on 24 March 2011; presented to the House on 25 March 2011) 
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Appendix C Report on the audit of the President of the Treasury Board’s Anmua/ Report to Parliament 
on the Tabling of Crown Corporations’ Reports 2011 


Tablings in Parliament for parent Crown corporations: Annual reports and summaries of corporate plans and budgets 


Section 152 of the Financial Administration Act (the Act) requires the President of the Treasury Board 
to lay before each House of Parliament a report on the timing of the tabling, by appropriate ministers, 
of annual reports and summaries of corporate plans and of budgets of Crown corporations. This report 
must be tabled by 31 December. 


The Act requires the Auditor General of Canada to audit the accuracy of the President of the Treasury 
Board’s report on the timing of the tabling and to present the results in his annual report to the House 
of Commons. 


At the time that our annual report was going to print, we were unable to include the results of the above 
audit, since the President of the Treasury Board’s report had not yet been finalized. The auditor’s report, 
which is required by the Act, will therefore be included in the next Report of the Auditor General of 
Canada to the House of Commons. It will also be appended to this year’s report of the President of the 
Treasury Board. 
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Appendix D Costs of Crown corporation audits conducted by the Office of the Auditor General of Canada 


The Office is required, under section 147 of the Financial Administration Act, to disclose its costs incurred 
in preparing annual audit (Exhibit D.1) and special examination reports on Crown corporations. 


An annual audit report includes an opinion on a corporation’s financial statements and on its compliance 
with specified authorities. It may also include reporting on any other matter deemed significant. A special 
examination determines whether a corporation’s financial and management control and information 
systems and its management practices provide reasonable assurance that assets have been safeguarded 
and controlled; financial, human, and physical resources have been managed economically and efficiently; 
and operations have been carried out effectively. 


In 2010-11, the Office completed the special examination of four Crown corporations. 
The costs incurred are in the following table: 


Canadian Dairy Commission $679,967 
Freshwater Fish Marketing Corporation $767,781 
National Arts Centre Corporation $822,932 
Telefilm Canada $732,825 
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Exhibit D.1 Cost of preparing annual audit reports for fiscal years ending on or before 31 March 2011 


Atlantic Pilotage Authority 


31.12.10 


Atomic Energy of Canada Limited (joint audit) 
Blue Water Bridge Authority 


Business Development Bank of Canada (joint audit) 


109,491 


310314 488,501 
31.08.10 149,968 
S103.1 522,676 


Canada Council for the Arts 3120S at 249,403 
Canada Deposit Insurance Corporation i i SieO3, bt 189,459 
Canada Development Investment Corporation (joint audit) SAG 2 LO 169,192 
Canada Employment Insurance Financing Board SiO3ia 88,470 
Canada Hibernia Holding Corporation (joint audit) Sl1Z27400 93,978 
Canada Lands Company Limited BOs | 850,788 
Canada Mortgage and Housing Corporation (joint audit) oulelecale 522,749 
Canada Post Corporation (joint audit) SIA 1O 705,872 
Canadian Air Transport Security Authority S108 1c 490,106 
Canadian Broadcasting Corporation ome OS palit 904,937 
Canadian Commercial Corporation SOS 1a 228,103 
Canadian Dairy Commission 3150710 238,668 
Canadian Museum for Human Rights Sic0sst 154,277 
Canadian Museum of Civilization 303.41 142,013 
Canadian Museum of Immigration at Pier 21 SL03;1 1 185,605 
Canadian Museum of Nature S031 146,668 
Canadian Race Relations Foundation 31.034 154,375 
Canadian Tourism Commission Set 2L0 364,452 
Defence Construction (1951) Limited Fo 1 Of jl 147,755 
Enterprise Cape Breton Corporation 31.03; 11 393,428 
Export Development Canada Sle Oo 1,032,923 
Farm Credit Canada SOS ava 769,743 
Federal Bridge Corporation Limited, The SU.03411 155,685 
First Nations Statistical Institute* | N/A | N/A 
Freshwater Fish Marketing Corporation 30.04.10 235,7 US 
Great Lakes Pilotage Authority oi, 12. 10 157,376 
International Development Research Centre 31 03. 11 207,368 
Jacques Cartier and Champlain Bridges Incorporated, The 31.03.11 ___ 197,620 
Laurentian Pilotage Authority 31.12.10 108,625 
Marine Atlantic Inc. 31.03.11 | 403,298 
National Arts Centre Corporation ; : 31.08.10 | _ 319,691 — 
National Capital Commission : ; : 31.03.11 | 328,395 
~ National Gallery of Canada ee 31.03.11 226,671 
BL 03. 11 173, 013 


National Museum of Science and Technology _ 


*The First Nations Statistical Institute's annual audit report for the fiscal year ended 31 March 2011 has not been started due to entity delays 


in completing the reports for the fiscal years ended 31 March 2008, 2009, and 2010. 
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Exhibit D.1 Cost of preparing annual audit reports for fiscal years ending on or before 31 March 2011 (continued) 


"3.05.1 ee 


231,738 


Old Port of Montréal CoIporaige Inc. 

Pacific Pilotage Authority Sie Zao 95,264 
Parc Downsview Park Inc. SLOSst 20 Ooi, 
Public Sector Pension Investment Board (joint audit) Sues sel 475,661 
PPP Canada Inc. (joint audit) Si08el1 115,682 
Ridley Terminals Inc. SUEZ © Z5Do220 
Royal Canadian Mint al wmeetiote 662,113 
Seaway International Bridge Corporation Ltd., The SlROSainl 96,775 
Standards Council of Canada 31.03.11 105,310 
Telefilm Canada Sl0sait 227,637 
VIA Rail Canada Inc. SIRUZEO 814,050 
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